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s uni AlfY 



This study invest 1 rat rd t hr> effect of using different refrac- 
tion modols upon the second order astronomic latitudes obtained from 
observed stars. Tho throo refraction models used wore those of B^ldinA , 
Garfinkol And tho Unitod States Coast and Geodetic Survey. It was 
found whon using tho method of observing a north and a south star to 
obtain tho final latitude, that no significant difference existed in 
the final combined values of latitude using any of tho refraction 
modols. However, tho difference betwocn tho north star and south star 
values of latitude were slightly larger using tho Garfinkel Model than 
whon the other two modols wore used. 
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PURPOSE AND SCOPE 

The major investigation undertaken in this thosis is tho exami- 
nation of tho effect of tho rofraction values given by three solected 
refraction models upon tho astronomical latitudo detormined from a sot 
of star observations at a given station. Tho throo rofraction modols 
which were used aro those of tho United Statos Coast and Gcodotic 
Survey, Baldini and Garfinkol The primary objective of 

the projoct has been to determine whether any significant difference in 
tho final value of the astronomic latitude exists whon oach of tho 
refraction modols is used. This would also include a ry effoct on the 
standard error of tho moan latitudo for oach station. Tho next objec- 
tive of the study was to determine if any model was sufficiently 
accurate to enable tho observer to obtain his latitudo position by 
observing one single star, oithor north or south, in place of tho current 
method of observing both a north and south star. Finally, it was attempt- 
ed to suggest a method which would includo an adjustment of tho ro- 
fraction coefficients from the observed data at a station, Tho Garfinkol 
model was tho modol used for this adjustment project. 

The observed data for thirtoon second order latitudo stations in 
Southwestern Ohio were used for this investigation. Each of these 
stations was reduced using oach refraction modol and tho results com- 
pared. This is listed in tablo 4, The IBM 7094 electronic computer was 
used for these reductions and the programs woro written in the SCATRAN 
language. 

In addition to the actual station roductions various aspects of 
the refraction models wore investigated to dotormine tho offocts caused 
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by tho variation of certain assumed values or constants upon tho computed 
refraction. This was performed on tho Baldini modol to dotormino tho 
effects of the relative humidity and the wave length of the observed 
light and on tho Garfinkol modol for cortain constants. Tho refraction 
values computed by each of tho throe models for various zonith distances 
were compared for a rango of temporaturo and prossuro combinations for 
selected conditions. 

This roport will bo generally divided into fivo main soctions 
in addition to this one. The second section will contain some basic 
definitions and describo selected classical theorios of tho variation 
of the earth's atmosphere. Tho third section Will briefly describo the 
basic assumptions that are made for tho throe refraction models and 
list their final computation equations. This section will also include 
the investigations performed on the various models and the results of 
the comparison of their refraction values. The fourth section will 
deal with the latitudo reduction mothods used in this study and tho 
results obtained. The fifth major section will be devoted to tho 
adjustment procedures and rosults obtained using the station data and a 
form of the Garfinkel Refraction Equation. The final section will serve 
as a summary and a statement of the major conclusions of the inves- 
tigation. 
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CHAPTER 2 



BACKGROUND 

2.1 General 

For this paper, astronomic refraction will bo dofinod as tho 
apparont displacement of an object that results from light rays from a 
sourco outsido tho atmosphere being bont in passing through tho atmos- 
phor©.(6) Tho measure of this quantity is tho change produced in tho 
direction of tho light rays. Tho total amount of this chango depends 
on the donsity of the various strata of air through which tho light rays 

pass and upon the direction of these strata with respoct to the vortical. 

( 2 , 7 ) 

It is tho consideration of tho constitution of tho atmosphero 
which givos riso to tho difforoncos in tho various proposod refraction 
models. The atmosphero of tho earth is known to bo more donso noar the 
surface of the earth and gradually decreases in donsity to its upper 
limit. For our purposos tho upper limit of tho atmosphere will bo tho 
point at which tho donsity is so low that its offoct on a ray of light 
is negligible. Since this condition causos tho ray of light to always 
pass from a medium of lower donsity to ono of higher density, tho result 
will bo that all observed objocts will appear to bo highor abovo tho 
horizon than they actually are. Figure 1 shows the path of a ray of 
light from tho time it enters tho atmosphore until it is soon by an 
observer. (2,6,7) 

The index of refraction for a given modia of constant donsity 
shall b© definod hero as tho ratio of tho sino of tho angle of incidonce 
to the sine of the anglo of rofraction, when those anglos are moasured 
with respect to a normal to the boundary surface whoro tho light ray 
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Earth 

Surface 



R = Refraction 

Z = Observed Zenith Distance 
o 

Z = True Zenith Distance 

Figure 1. Refraction Caused by the Earth's Atmosphere 
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passos into the given media from a vacuum. Tho angle of incidonco is 
measured at tho boundary surfaco bofore tho ray passos into tho media 
and the angle of refraction is measurod after it has passed through tho 
boundary surfaco. 

2.2 Classical Theorios of tho Variation of the Earth's Atmosphere. 

Tho donsity variation of tho oarth's atmosphoro from tho surfaco 
to tho upper limit is assumed to bo subject to the conditions of 
equilibrium. Therefore, it can bo generally stated that tho donsity 
varies with temperature and pressure. Sinco tho astronomic refraction 
is dependent on the density, it may also bo related to tho variation of 
the temperature and pressure of the atmosphoro. The various rofraction 
tables which have been constructed for astronomical use rost upon 
difforont expressions for tho variation of the donsity of tho atmos- 
phere with the altitude. Four of tho classical thoories for this vari- 
ation will bo reviewed in tho following paragraphs. 

The first real attempt to describe tho variation of the atmos- 
phere was put forth by Nowton. This hypothesis assumed a constant 
temperature at all altitudes. Tho relation has been discarded in modern 

timos and therefore this theory should be considered only in an histor- 

(7) 

ical sense, 

Bessel proposed that the density could be expressed in terms of 
altitude only and therefore assumod no law of temperature, Tho ratio of 
the densities at two altitudos is expressed as 




whore Q is tho density at altitude h and the subscript one refers to tho 
conditions of the base station. K is a constant factor used for com- 
putations ( K=0. 
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Ivory put forth the hypothesis that tho tomporaturo diminishes 
at a uniform rat© with tho height at all altitudes. This roprooonts 
tho ratio of tho don3itios as 

^ - O- 

where 

T= T» and ~ ^ 

In this rolatlon h 0 is th© altitude at which tho atmosphere would tori- 
nato, Q is a constant factor usod to rolato tho tomporaturo (T) at a 
given altitude (h) to tho tomporaturo at tho baso station (Tj), and 
the exponent y is ©xprossod in toms of th© height of tho baso station 
(hj ) and . 



h n h n 

AA\ \ = -il = O 

^ hi Vi* 

whoro tho value of V Is a constant, doponding on tho elasticity of air 
at a given tomporaturo and density, (?) 

Iho last classical hypothosi3 to bo givon horo states that tho 
temperature diminishes by a constant fraction of its absolute amount for 
every unit increase in altitudo. This is in tho fona of a constant 
geometrical progression which may be represented, in toms of tho ratio 
of densities, as 

3_ - ^-\T 



where \T ^ l') - p V, 

The small g roprooonts the value of gravity and all tho other toms havo 
been proviouoly doflnod. (?) 

Thoso theorioo havo boon proeonted as a background for tho 
dlsoussion of tho specific rofraotion modols with which wo cro con- 
cerned in the next section. 
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CHAPTER 3 



REFRACnON MODELS INVESTIGATED FOR TOTS PROJECT 

3.1 General 

The descriptions of tho throe rofraction models which follow 
are not intendod to bo dotailod discussions of the procedures used to 
derive the refraction equations. Rather, they simply attempt to give 
the basic assumptions upon which the models are based and the final 
forms of the rofraction equations used for computations , It will be 
attempted to point out any limitations introduced into the models and 
any similarities which may exist. 

The computer programs used for the practical computations with 
each refraction model will be generally described after tho basic 
discussion of each model. The statements listings and required input 
data for each of the programs covered may be found in Appondix I, 

To save repetition with each refraction modol, some basic terms 
common to all threo models will be defined hero. The quantity R will 
always represent the astronomic refraction. Z will be tho zonith 
distanco corrected for refraction and Zo will represent tho obsorvod 
zenith distance. T and P will represent tho absoluto tempera turo and 
pressure respectively. When these last two quantities appear with the 
subscript (o) they will refer to the conditions at the observation 
station. All othor quantities will be defined when thoy appoar in the 
discussion for each model. 

3.2 The United Stato3 Coast and Goodotic Survoy Rofraction Model 

This refraction model is based on a thoorotical background which 
is combined with physical data to produce a proposed composition of the 
atmosphere. This model follows that proposed by Willis and is 
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generally based upon the theories of Newcomb. (?) 

First, the pressure and temperature of the atmosphoro aro 
assumod to vary with rospoct to the density in the relation given for 



a perfect gas. This relationship is given as 

- Asl 

T 
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where i 3 is tho density and k is a constant dopending on the units used 
and the gas in question. Willis assumod that tho temperature and 
prossure in free air varies according to tho relation that 
(y.) = 0.670.0. 5295(^0. 2625<£ > 2 

whore tho relative temporaturo at a point is given as a function of the 
relative pressure. The rolativo pressure is known to be closely ro- 
tated to the mass heights of tho atmosphere. The density of tho atmos- 
phere at a point could then bo computed after tho pressure and temper- 
ature were obtainod. (3#l8) 

The derivations for this model are based upon tho assumption 
that tho atmosphere takes tho form of concontric spherical surfaces. 
These may bo described as horizontal strata which are of equal density 
and follow level surfaces. Any ollipticity of the actual physical sur- 
face is neglected. ( 3»H ) 

Up to a zenith distance of 60 degrees a first approximation of 

the refraction formula may be given as 

n kB . 

R= Tjr- tan X 

where B is the barometric pressure at tho point of observation and k is 
a constant whose value depends on tho units of temporaturo and prossure. 
For a more accurate value of the refraction and for zonith distances 
over 60 degrees more terms must bo addod to this basic expression. 

Tho form used by the Coast and Geodotic Survoy is 
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R- ■< ( tan Z)fl-(vT-Jj<#Osec^Z) 

1+^- *- ' 

which oxtonds tho reliable application of the refraction formula to a . 
zonith distanco of 70 dogroos or noro. Tho quantity -pppj nearly oquals 
the rofractivo index of tho air. Tho quantity V“ is based on tho 
radiuo of curvaturo of tho strata and proportios of tho atmoephoro at 
tho station. Ihio is roughly equal to . This formula may bo 

expressed as a part of an infinito series which i3 given as 

R (tan Z) (1-C- sec^ + C. sec 4 Z - C, sec 6 Z + — ). 

1+oc- 2 4 o 

Tho C£ coefficient ia tho quantity \T -2x5- and tho remaining coofficionts 
dopent on tho constitution of tho atmosphere. Even after those coeffi- 
cients havo boon dotemined for an assumed state* of things, tho oorlos 
coaso3 to converge before tho horizon is roachcd. It may bo used 
practically for computations up to a zenith distanco of 83 or 84 degroos. 
For the practical computation of tho refraction a logarithmic 
form of the above rofraction formula is usod, Thio may bo represented 
as 

loS 10 R " 105 10 (tan + log io ( §"* + lo S 10 (1 " C 2 sec2z) 

whore tho quantities represented by tho logarithms are terms of tho 
refraction formula given above, § and 'ft' roprosont departures from 
tho standard atno3phoric conditions of pressure and temporaturo. Tho 
term containing C2 is added as a correction to tho ^ /( 1-4-c*) torn which 
varies slowly with modorato zonith distance. However, whon thio term 
is introduced it does not allow tho wholo rofraction equation to bo 
taken as proportional to temporaturo and pressure. Thio is taken into 
account by tho ()* -l)(X + -^j) torn in tho expression. This last torm 
does not bocomo important except at tho largor zenith distances, (3* 

Tho tables published for this refraction model wore derived for 
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a pressure of 760 millimeters, a temperature of 10 degrees centigrade, 
a wave length for yollow light of 0,578 micron and a relative humidity 
of 60 per cent. Correction tables are provided for differences in the 
observed temperature and pressure from these selected values. These 
tables were used in the computations for this investigation. (3 t ll) 

The computer program used for the Coast and Geodetic Survey Re- 
fraction Model was fairly simple and straightforward. The refraction 

values givon in Table V of the U, S. Coast and Geodetic Survey Publi- 
(3) 

cation No. 237 were read into the main program for every ten minutes 
of arc. Since these values were computed for a given set of conditions, 
correction coefficients for temperature and pressure variations had to 
be used. These were taken from Tables VI and VII of the above publi- 
cation. These correction coefficients were introduced into the sub- 
program using literal statements. These values were limited to a 
reasonable range of temperature and pressure for the reduction of the 
observed data. The range of these values was expanded for the compari- 
son of the three refraction models at various extremes of temperature 
and pressure. The subprogram was made up of three search loops which 
computed the correction values for the given temperature and pressure 
and the refraction value for the observed zenith distance. All of these 
factors were then multiplied together to obtain the astronomic refrac- 
tion. 

3.3 The Baldlni Refraction Model 

The refraction equations in this model are derived for an 
object Inside the atmosphere but the final form of the equations is 
such that they will hold for an object at infinity. Since it is assum- 
ed that all stars are at infinity, this model may be used to compute 
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astronomic refraction 



For his derivations Baldini assumed a spherical earth with an 
atmosphere arranged in sphorical layors. Ho adopted tho conditions 
that tho altitudo of the atmosphoro would always bo loss that 64 
kilomotors and the observed zenith distance would bo loss than 75 de- 
grees. This altitude limit represents the maximum height of strata ovor 
which tho refracting power of tho atmosphoro is assumed to be zero. 

In this model Baldini defines n as tho index of refraction of 
one layer in the atmosphere and n + dn as tho index of refraction of 
the next layer. Ho states his basic rofraction equation in tho form 



CC 



tan Z 



dn 

n 



Sl "n \ 11 

where Z is the zenith distance of tho object in tho first layer of the 
atmosphere. with an index of refraction n. To solve this equation for 
the rofraction expressions for dn/n and tan Z aro required. To obtain 
theso, since both are a function of the density of the atmosphere, 

Baldini used the relationship for density at various altitudes that 

P _ - _h_ 

- e no 

in which h is a constant and ^o is the density at tho observer's loca- 
o 

tion on the earth's surface. Q is the density at an altitude h in the 
atmosphere. After determining a value for h this expression bocomes 



Q = e. 



■ O. \OR3- V\ 



where h is given in kilometors. This expression is similar in form to 
the relation proposed by Bessel. 

Using theso relationships tho final form of the refraction 
equation for an object at infinity becomes 

R ■ Ao(no-l) tan Zo + A^(no-l) tan^Zo + A2(no-l) tan^Zo 
with the quantities A Q , Aj, and A^ givon as 
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H 







Ao = +0.9982? 



Aj = -0.00130 

A 2 = +0.000006 

from tho derivations. The factor ("H -1 ) • withTbo as the indox of refrac- 
tion at the observer's station, is computed from the equation of Barrel 
and Soarsu This computation is done in two stops with a preliminary com- 
putation being performed for the refractivo index of standard air at 
optical frequencies. This value, designated as”8g, is obtained using 
the formula 

(Trig -1) i 0 7 = 2.87604 + ^.288 + 

> 2 >, 4 

where represents the light wave length in microns. This is then con- 
verted to the ambient conditions using the relationship that 

(•ho -1 ) =4Tg-l _P_ _ 0.000000055 e 

l+<&760 1+-<CX 

whero e is the partial pressure of water vapor in tho air, is the 
coefficient of expansion of air (O.OO 367 ) and X. is tho temperature in 
degrees centigrade. The quantity e. is obtained from tho relative humid- 
ity and a table of saturated vapor pressures for various tempera turos. ^ ) 
The computer program used for this rofraction model is self- 
contained in its own subprogram. Tho computations using the Baldini Ro- 
fraction Model wore all done for an assumed relative humidity of 60 per 
cent and a light wave length of 0.578 micron. A table of saturated vapor 
pressures for selected temporatures was entered in the subprogram 
through the use of a literals statement. A search loop similar to those 
used in the Coast and Geodetic Survey Model program was employod to 
obtain the saturated value of Q. at the desirod temperaturo. This 
value was modified for use in the program by multiplying it by tho rela- 
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tivo humidity. The quantity (n Q -1) was thon computed and the astro- 
nomic refraction obtained from the final refraction formula given above. 

3.31 Effects of Relative Humidity and Light Wave Length 
on the Baldini Hodol 

An investigation was run on this subroutine to dotormine tho 
effect of differences in the relative* humidity and tho wave length of 
light used for the observation. It was found that tho refraction varied 
very little for a range of relative humidity from 10 to 90 per cont. 

This was for a zenith distance range of 0 to 85 degrees with comparisons 
being made at ovory fivo degreos of zenith distance. Those results are 
includod in Table 1 at the end of this subsection. 

The effect of the wave length of light was examined for wave 
lengths ranging from 0,4 75 to 0,693 micron which is tho rango of the 
visible spoctrum. There is a larger variation due to tho wave length 
effect than was found with tho relative humidity. The differences wore 
as large as three seconds of arc at a zenith distanco of 75 degrees. 

The results of this investigation are includod in Table 2 at tho ond of 
this subsection. 

From tho results of these investigations it has boon assuraod 
that tho values used in tho computation program wore sufficiently accu- 
rate as there was very little variation in the rofraction duo to the 
relative humidity and noarly all observablo stars omit light in the 
yellow wave length region of 0.578 micron. 
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EFFECTS OF RELATIVE HUMIDITY 
ON THE 

BALDINI REFRACTION MODEL 
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Length - 0.578 Microns 



EFFECTS OF WAVE LENGTH 
ON THE 

BALDINT REFRACTION MODEL 
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3.4 The Garfinkol Refraction Model 

The Garfinkel Modol is the only one of tho throo used in this in- 
vestigation which was designod to give refraction values for zenith dis- 
tances up to and greater than 90 dogroos. Thore are some variations in 
the final form of the refraction equation for this modol among tho 
agencies which uso it. Two of thoso variations will bo given after a 
discussion of tho original form derivod by Garfinkol. 

Garfinkel selects a modol for tho variation of the donsity of 
tho atmosphoro that is similar to tho modol suggested by Ivory. In a 
modol of this type the temperature diminishes with a uniform rato re- 
gardloss of the altitudo. This type of modol is called a polytropic 
model. (5) 

With the perfect gas law assumed to hold for the earth's atmos- 
phere, the density relationship is given in its final form as 




where 9 is the relative density of tho atmosphere at an altitudo y 
above tho observation station. The quantity n is called the polytropic 
indox. The relative tomperaturo is considered to diminish at a constant 
rate to the point where tho altitudo is y^. Above this point the rola- 
tive temperature is considorod to be zero. This relation may be given 



by 



T'= - 



n+1 



where T' is the constant rate at which the relative temperature de- 
creases. The relative tempera ture is then given as 



T= 1 2L. 

n+1 

where the condition at the observation station would be 

T = 1 and y = 0. 
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It nay be soon that using this relationship for the temporaturo the 
value y^ must satisfy the relation 

y l - » + 1 

for th© relativo temperature to equal zero at that altitude. 

From this polytropic relationship Garfinkol develops a formula 
in which he exprossos the refraction as a power series which is a func- 
tion of the observod zonith distance. Ibis equation is of the form 

FfeT c 2w( B 0 +B 1 W+B 2 W 2 +B 3 w3+ ) 

for zenith distances that are less than 90 degroes. Tho terms which 
make up this series will be explained in the following paragraphs. ( 5) 

In the derivation of tho final form of this refraction equation 
there are some constants which appear. The most important of thoso are 
called K 0 , 0 and V 0 . Those are affected by local conditions at the 

observing station as well as the choico of a value for the polytropic 
index. If a standard set of conditions is chosen and thoso valuos com- 
puted for them, these values can be converted to the conditions at the 
observation station by temperature and pressure relationships. 

Some factors must be defined before th© above three constants 
can be computed. When the index of refraction is denoted by/U~ 0 at 
the observation station, a quantity cc Q is defined from 

= ( a o-l )/Z"b • 

From this factor a value C 0 is defined as 

C o = ^ 0 (1+ °o^ * 

Finally, a factor S c is defined by 

S 0 = 

^0 

where Ho is the "height of the homegonous atmosphere for an ideal gas" 

and r is the radius of the earth to tho observer's station. This 
o 
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height, Ho, is the height to which the atmosphoro would oxtond abovo 
the oarth if its density wero the oamo as at the point of observation 
and tho prossuro w»3 equal to tho observed prossuro. (3. 5) 

Using thc 3 o values from tho preceding paragraph, tho constants 
for tho standard sot of conditions can now bo dofinod, Ihoso values are 
given a double subscript to Indicato that thoy aro dorived for oortain 
stipulated conditions and at standard temporaturo and prossuro, Thoso 
constants aro defined as 

Uo *d s o3"’ i (n ♦ O'* 

Boo - 2'-do>oo 2 <" + 1 
K oo - 2 C„„ !foo *»’’*■ 

using a chosen valuo for n, Tho variation of all tho factors dofinod 
for tho standard conditions, oxcopt tomporaturo and pressure, is so 
snail that it mako3 no significant chango3 in tho values of thoso con- 
stants from station to station, Theroforo, working valu©3 for those 
constants nay bo obtained at an ob3orving station from 



Yo -fco 1 
B o - Boo W 
K « K co T P\-1 



where V/ is tho ’’weather factor” and is obtainod from 

Wo F °/to2 



whero P Q and To represent tho ratios of tho pressure and temporature at 
the observing station to those standard conditions for which the values 
were originally computed, Garfinkol computod those constants using a 
valuo of 5.000 for n and obtained 

Yoo » 8.1578 

^>00 = 0.03916 
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Koo = **952" 

at a tomporature of 0 degreos centigrade and a pressure of 760 milli- 
metors. (5) 

From these constants the B factors of the powor sories are de- 



fined as 



Koo^ oo'vi i 

whero ^ is a function of an angle theta. This angle is dorived from 

the observed zenith distance by the relationship that 

_1 

cotQ= VT 0 -i! cot21 0 . 

^ may also be expressed in terms of a power series of tan®/®^-^ 

The refraction equation may be written for zenith distances 
greater than 90 degrees using tho above quantities. From those factors 
two new values are defined. They are 

c i - 2 B -- c ° s2 % +1 



f / 2 



and 

V - lo tan 2 e /2 
To z 

The refraction for a zonith distance greater than 90 degroos is then 
obtained from the formula 

Kb T 0 2 V(C 0 tCiV+C 2 V 2 +C3V 3 + ) 

which is also expressed in the form of a powor series, (-5) 

The program which was used for most of the rofraction computa- 
tions wa3 that of the Garfinkel Modol as programmed by Aberdeen. This 
program was translated from the FORTRAN II version into the SCATRAN Lan- 
guage. It basically follows tho summations givon by Garfinkel in his 
article to obtain the factors, Tho rofraction is then computed 
using Garfinkel 1 s power series formula as given above. This program 
uses updated factors for Garfinkel* s basic constants. These are 
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n= 4,256 
Koo= 4472.8 
$ 00 = 0.044385 
Voo= 8.7137 

which are computed for the samo standard conditions of temperature and 
pressure as those of the Garfinkel article. In a comparison of the 
program using the two sots of constants, it was found that this newer 
set of valuos gives a smaller amount of rofraction for oach selected 
zenith distance than whon the original Garfinkel set was used. This 
difference has a magnitude of about 1,5 soconds at a zenith distance of 
75 degrees. This comparison is containod in Appendix I. Tho Aberdeen 
constants were used for all reductions and model comparisons in this 
project. 

Another form of the Garfinkel Rofraction Model is used by the 
Coast and Geodetic Survey. This mothod computes the refraction from 
the formula 

R - To^ (Nj ton B ^2 + N? tan 3 B/2 + tan 5 B/2 
+ tan' ) 

where the constants are given the valuos 

N* = 1050.61030 
n 2 = 706.11502 
= 262.06086 
N 4 = 142.67293 

and the angle B is the same as the angle 0 in the Garfinkel articlo 
formulas. The X factor used in this method is tho same as that of the 
Aberdeen program. 

This refraction method was also programmed and used in some 
reductions. It gives slightly lower refraction valuo3 for zenith 
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distances up to 70 degrees and slightly highor values for the zenith 
distances ovor this value when compared to the Abordeon program. The 
difference amounted to about 0.02 seconds of arc at a zenith distanco of 
75 degrees. The advantage of this program over that of Aberdeon is 
that it is much shorter and uses about one-fourth of tho computer time 
for tho same purposes. Appendix I contains a comparison of the results 
of this subroutine with the Abordeon valuos for one selected sot of 
temperature and pressure conditions. 

Another vorsion of the Garfinkol rofraction equation is that 
used by the Aeronautical Chart and Information Center (ACIC).(^ This 

obtains the refraction from tho equation 

R ■ tanO + N2 tan^o+ N3 tan^Q 

in which the angle 0 is tho same as given in tho Garfinkol article. 

This system uses a Y value of 8.1578 which is tho same as Garfinkol' s 
original value. This form was programmed by using the values given in 
Garfinkol' s tables for the various theta angles as tho N factors in the 
refraction equation. This vorsion was used only as a comparison for tho 
other programs given abovo. It is included in the comparison givon in 
Appendix I. 

This form of tho refraction equation was used as a basis for the 
adjustment program discussed in section 5. N factors woro assigned for 
the purposes of the adjustment and a Y factor of 8.7137 was used in that 
program. 

3.41 Commonts on the Garfinkol Program 

The Garfinkel model was found to givo tho greatest difforonco 
between tho computed values of latitude, from tho observed north and 
south stars, of the three refraction models usod. The Abordoon sub- 
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routino was them used witli difforont valuos of Y qo but holding all other 
factors constant. This would bo equivalent to changing tho coeffi- 
cients of tho K qo and ^> 0Q terms which aro computod from Thoso co- 

officients aro given as 2.0 by Garfinkel. It was found that a highor Y 
under thoso conditions gave loss difforonco botweon the north and south 
star computed latitude values for all stations examined. There worn 
difforont values of tho factor which gave tho minimum spread for each 
station but they all wore higher than tho soloctod valuo. Thoso values 
ranged from 8.78 to 8.88 for tho minimum closure on each station. How- 
over, tho values of tho factor over tho 8.78 lovol had tho effect of re- 
versing tho error in somo stations whon it was used to roduco all the 
observed stations. Thoroforo, any valuos higher than this ono would 
bo of questionable merit. This offoct was to basically switch tho 
positions of the north and south latitudes whon comparod to tho com- 
bined valuo. This method of forcing loss difforonco botwoon the north 
and south stars is not given as a proof of any weaknoss in tho modol 
but is considered worthy of note sinco there is an indication that an 
adjustment in tho constant factors for tho Kq 0 and ^> 0Q valuos by a 
slight amount might result in a moro accurato refraction modol. This 
would bo associated with a now higher valuo for the gamma constant that 
would bo somewhere between the Aberdoon value and tho 8,78 quantity 
montionod abovo. Much moro study and research would bo roquirod boforo 
it could bo decided whether the noted changos were either roquirod or 
practicable. Tho idea is put forth as a possible area for future re- 
search but should not be accopted as anything moro meaningful than that. 

3.5 Comparison of the Refraction Valuos Obtainod from tho Throe 
Refraction Models. 

A comparison of the rofraction valuos for the Baldini Model, 



22 



the Garfinkol Modol and tho Coast and Goodotic Survey Model was made for 
various conditions and extremes of temporaturo and pressure. These com- 
parisons wore mado for a standard relativo humidity of 60 por cent, 
light wave length of 0,578 micron and over a range of zenith distances 
from 0 to 90 degrees. It should be noted that tho form of the Baldini 
model goes to infinity at 90 degrees and so no comparison can be made 
there. 

It should also be noted in theso comparisons that the Baldini 
Modol was designed to bo used only for observed zonith distances up to 
75 degrees and that the series used for tho Coast and Geodetic Survey 
Model becomes questionable after a zenith distance of 83 or 84 degrees 
is reached. ihe comparisons wore mado though to soo how close 

the models did come. Table 3 lists the refraction values for various 
combinations of temperature and pressuro and also tho difference in the 
obtained values between the models. This is dono using the Coast and 
Geodetic Survey Model as an arbitrary standard with which to compare 
the other two models. 

The comparisons show that the Baldini and Coast and Goodetic 
Survey Models tend to agree more closoly throughout the range of tem- 
perature and pressure than tho Garfinkel and tho Coast and Geodetic 
Survey Models. However, at higher temperatures the refraction values 
given by the Baldini Modol tend to become lower than those given by the 
Coast and Geodetic Survey Modol, At lower temperatures those values 
are highor than those given by the Coast and Goodetic Survey Model. 

The widest variation between the Baldini and Garfinkel Models occur at 
the higher tempera tures. The relation betweon the Garfinkol Model and 
the Coast and Geodetic Survey Model at the higher zenith distances. 
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over 85 degrees, varies quito sharply with Lho temperature. In this 
area the Garfinkel Model gives highor valuos of refraction at tho lowor 
temperatures than the Coast and Goodotic Survey Modol and lowor ones at 
the highor temperatures. It was found generally that tho refraction 
differences appear to bo more offectod by the variation of tomperature 
than those of pressure. All the modols give fairly close valuos for the 
refraction up to a zenith distance of about 50 dogroos in which area tho 
difference is about 0.3 seconds. 

For the comparisons in Table 3 the valuos of the refraction were 
rounded to the nearest 0.1 second. This is done since the tables for 
the Coast and Geodetic Survey Model are only given to this accuracy. 
Since the errors in this range are within the acceptable limits for 
second ordor work this factor does not tend to affect the results of 
this investigation. The Aberdeen subroutine was used for the Garfinkel 
Refraction Model in this comparison. 
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TABLE 3 



REFRACTION COMPARISONS 
FOR THE 

THREE REFRACTION MODELS 



Zenith 

Distance 

(donroos) 


Coast 

and 

Goodotic 

Survey 


Baldini 


Garfinkol 


Difference 
and Geodetic 
Baldini 


from Coast 
Survey Model 
Garfinkol 


Temporature -5 C Pressure 711 mm Refraction in seconds 




0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


5.0 


5.0 


5.0 


0.0 


0.0 


10 


10.1 


10.1 


10.1 


0.0 


0.0 


15 


15.3 


15.4 


15.4 


0.1 


0.1 


20 


20.9 


20.9 


20.9 


0.0 


0.0 


25 


26.7 


26.8 


26.8 


0.1 


0.1 


30 


33.0 


33.1 


33.2 


0.1 


0.2 


35 


40.0 


40.1 


40.2 


0.1 


0.2 


40 


48.0 


48.1 


48.2 


0.1 


0.2 


45 


57.2 


57.3 


57.4 


0.1 


0.2 


50 


68.1 


68.3 


68.4 


0.2 


0.3 


55 


81.5 


81.8 


81.9 


0.3 


0.4 


60 


98.9 


99.0 


99.2 


0.2 


0.4 


65 


122.1 


122.4 


122.6 


0.3 


0.5 


70 


156.0 


156.2 


156.6 


0.2 


0.6 


75 


210.3 


210.6 


211.3 


0.3 


1.0 


80 


313.6 


313.9 


315.4 


0.3 


1.8 


85 


581.6 


611.7 


585.8 


30.1 


4.2 


90 


2035.6 




2051.2 


oO 


15.6 


Temperature -5°C Pressure 737 mm 


Refraction in seconds 




0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


5.2 


5.2 


5.2 


0.0 


0.0 


10 


10.4 


10.5 


10.5 


0.1 


0.1 


15 


15.9 


15.9 


15.9 


0.0 


0.1 


20 


21.6 


21.7 


21.7 


0.1 


0.1 


25 


27.7 


27.7 


27.8 


0.0 


0.1 


30 


34.2 


34.3 


34.4 


0.1 


0.2 


35 


41.5 


41.6 


41.7 


0.1 


0.2 


4 o 


49.8 


49.9 


49.9 


0.1 


0.1 


45 


59. 3 


59.4 


59.5 


0.1 


0.2 


50 


70.6 


70.8 


70.9 


0.2 


0.3 


55 


84.5 


84.7 


84.9 


0.2 


0.4 


60 


102.4 


102.7 


102,8 


0.3 


0.4 


65 


126.5 


126.8 


127.1 


0.3 


0.6 


70 


161.6 


161.9 


162.4 


0.3 


0.8 


75 


218.0 


218.3 


219.1 


0.3 


1.1 


80 


325.0 


325.4 


326.9 


0.4 


1.9 


85 


602.7 


634.1 


607.5 


30.1 


4.8 


90 


2109.6 


oc? 


2132.7 


00 


23.1 
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TABLE 3 (CON'T) 



REFRACTION COMPARISONS 
FOR THE 

THREE REFRACTION MODELS 



Zenith 


Coast 

and 


(Continued) 


Difference 


from Coast 


Distance 


Geodetic 




and Goodotic 


Survey Model 


(donroes) 


Survey 


Bald ini Garfinkol 


Baldini 


Garfinkol 



Tetnporature -5°C Pressure 760 mm Refraction in seconds 



0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


5.4 


5.4 


5.4 


0.0 


0.0 


10 


10.8 


10.8 


10.8 


0.0 


0.0 


15 


16.4 


16.4 


16.5 


0.0 


0.1 


20 


22.3 


22.3 


22.4 


0.0 


0.1 


25 


28.5 


28.6 


28.6 


0.1 


0.1 


30 


35.3 


35.4 


35.5 


0.1 


0.2 


35 


42.8 


42.9 


43.0 


0.1 


0.2 


40 


51.3 


51.4 


51.5 


0.1 


0.2 


45 


61.1 


61.3 


61.4 


• 0.2 


0.3 


50 


72.8 


73.0 


73.1 


0.2 


0.3 


55 


87.1 


87.4 


87.5 


0.3 


0.4 


60 


105.6 


105.9 


106.1 


0.3 


0.5 


65 


130.4 


130.8 


131.1 


0.4 


0.7 


70 


166.6 


167.0 


167.4 


0.4 


0.8 


75 


224.7 


225.1 


225.9 


0.4 


1.2 


80 


335.1 


335.5 


337.2 


0.4 


2.1 


85 


621.4 


653.9 


626.7 


32.5 


5.3 


90 


2174.8 


OO 


2205.3 




30.5 


Temperature -5°C 


Pressure 


787 mm Refraction in seconds 




0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


5.6 


5.6 


5.6 


0.0 


0.0 


10 


11.2 


11.2 


11.2 


0.0 . 


0.0 


15 


17.0 


17.0 


17.0 


0.0 


0.0 


20 


23.1 


23.1 


23.1 


0.0 


0.0 


25 


29.5 


29.6 


29.7 


0.1 


0.2 


30 


36.5 


36.7 


36.7 


0.2 


0.2 


35 


44.3 


44.5 


44.5 


0.2 


0.2 


40 


53.2 


53.3 


53.3 


0.1 


0.1 


45 


63.3 


63.5 


63.5 


0.2 


0.2 


50 


74.5 


75.6 


75.7 


0.1 


0.2 


55 


90.3 


90.5 


90.6 


0.2 


0.3 


60 


109.4 


109.6 


109.8 


0.2 


0.4 


65 


135.1 


135.5 


135.7 


0.4 


0.6 


70 


172.6 


172.9 


173.4 


0.3 


0.8 


75 


232.8 


233.1 


233.9 


0.3 


1.1 


80 


347.1 


347.5 


349.2 


0.4 


2.1 


85 


643.7 


677.2 


649.3 


33.5 


5.7 


90 


2253.1 




2291.1 


0 ^ 


38.0 
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TABLE 3 (CON»T) 



REFRACTION COMPARISONS 
FOR TUB 

THREE REFRACTION MODELS 



( Continued ) 



Zenith 
Distance 
(dofcrees ) 


Coast 

and 

Geodetic 

Survey 


Raldi ni 


Garfinkol 


Difference from Coast 
and Goodotic Survoy Model 
Paid ini Garfi nkol 


Temp ora turo 


i 10°C 


Pressure 711 


mm Refraction in seconds 


0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


4.8 


4.8 


4.8 


0.0 


0.0 


10 


9.5 


9.6 


9.6 


0.1 


0.1 


15 


14.5 


14.6 


14.6 


0.1 


0.1 


20 


19.7 


19.8 


19.8 


0.1 


0.1 


25 


25.3 


25.3 


25.4 


0.0 


0.1 


30 


31.3 


31.3 


31.4 


0.0 


0.1 


35 


37.9 


38.0 


38.1 


0.1 


0.2 


40 


45.5 


45.5 


45.6 


0.0 


0.1 


45 


54.2 


5 4.2 


54.4 


. 0.0 


0.2 


50 


64.5 


64.6 


64,8 


0.1 


0.3 


55 


77.2 


77.4 


77.5 


0.2 


0.3 


6o 


93.6 


93.7 


93.9 


0.1 


0.3 


65 


115.6 


115.8 


116.1 


0.2 


0.5 


70 


147.7 


147.8 


148.2 


0.1 


0.5 


75 


199.2 


199.2 


199.9 


0.0 


0.7 


80 


297.0 


297.0 


298.0 


0.0 


1.0 


85 


550.7 


578.8 


551.2 


28.1 


0.5 


90 


1927.7 


cx^ 


1874.3 




-53.4 


Temperature 


10 °C Pressure 737 i 


ran Refraction in seconds 




0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


4.9 


4.9 


4.9 


0.0 


0.0 


10 


9.9 


9.9 


9.9 


0.0 


0.0 


15 


15.0 


15.1 


15.1 


0.1 


0.1 


20 


20.5 


20.5 


20.5 


0.0 


0.0 


25 


26.2 


26.2 


26.3 


0.0 


0.1 


30 


32.4 


32.4 


32.6 


0.0 


0.2 


35 


39.3 


39.4 


39.5 


0.1 


0.2 


40 


47.1 


47.2 


47.3 


0.1 


0.2 


45 


56.2 


56.2 


56.3 


0.0 


0.1 


50 


66.8 


67.0 


67.1 


0.2 


0.1 


55 


80.0 


80.2 


80.4 


0.2 


0.4 


60 


97.0 


97.1 


97.4 


0.1 


0.4 


65 


119.8 


120.0 


120.3 


0.2 


0.5 


70 


153.1 


153.2 


153.7 


0.1 


0.5 


75 


206.4 


206.5 


207.3 


0.1 


0.9 


80 


307.8 


307.9 


309.0 


0.1 


1.2 


85 


570.7 


600.0 


571.6 


29.3 


0.9 


90 


1997.7 




1948.1 




-49.6 
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TABLE 3 (CON’T) 



REFRACTION COMPARISONS 
FOR THE 

THREE REFRACTION MODELS 



Zenith 
Distance 
(decrees ) 


Coast 

and 

Geodetic 

Survey 


(Continued) 
Baldini Garfinkcl 


Difference 
and Geodetic 
Baldini 


from Coast 
Survoy Modol 
Garfinkol 


Temporatur© 10 C Pressure 


760 


mm Rofrection in soconds 




0 


0.0 


0.0 




0.0 


0.0 


0.0 


5 


5.1 


5.1 




5.1 


0.0 


0.0 


10 


10.2 


10.2 




10.2 


0.0 


0.0 


15 


15.5 


15.6 




15.6 


0.1 


0.1 


20 


21.1 


21.1 




21.1 


0.0 


0.0 


25 


27.0 


27.1 




27.1 


0.1 


0.1 


30 


33.4 


33.5 




33.6 


0.1 


0.2 


35 


40.5 


ho. 6 




ho. 7 


0.1 


0.2 


ho 


48.6 


h8.7 




48.8 


0.1 


0.2 


45 


57.9 


58.0 




50.I 


. O.l 


0.2 


50 


68.9 


69.1 




69.2 


0.2 


0.3 


55 


82.5 


82.7 




82.9 


0.2 


0.4 


60 


100.0 


100.2 




100.4 


0.2 


0.4 


65 


123.5 


123.8 




124.1 


0.3 


0.6 


70 


157.8 


158.0 




158.5 


0.2 


0.7 


75 


212.8 


213.0 




213.7 


0.2 


0.9 


80 


317.3 


317.5 




318.6 


0.2 


1.3 


85 


5 88.4 


618.8 




5 89.7 


30.4 


1.3 


90 


2059.5 


oo 




2013.7 


oo 


-45.8 


Temperature 10 C Pressure 


787 


mm Refraction in soconds 




0 


0.0 


0.0 




0.0 


0.0 


0.0 


5 


5.3 


5.3 




5.3 


0.0 


0.0 


10 


10.6 


10.6 




10.6 


0.0 


0.0 


15 


16.1 


16.1 




16.1 


0.0 


0.0 


20 


21.9 


21.9 




21.9 


0.0 


0.0 


25 


28.0 


28.0 




28.1 


0.0 


0.1 


30 


34.6 


34.7 




34.8 


0.1 


0.2 


35 


42.0 


42.1 




42.2 


0.1 


0.2 


ho 


50.3 


50.4 




50.5 


0.1 


0.2 


45 


60.0 


60.0 




60.2 


0.0 


0.2 


50 


71.4 


71.5 




71.7 


0.1 


0.3 


55 


85.5 


85.6 




85.8 


0.1 


0.3 


6o 


103.6 


103.7 




104.0 


0.1 


0.4 


65 


127.9 


128.2 




128.5 


0.3 


0.6 


70 


163.5 


163.6 




164.1 


0.1 


0.5 


75 


220.5 


220.6 




221.3 


0.1 


0.8 


80 


328.7 


328.8 




330.0 


0.1 


1.2 


85 


609.6 


640.8 




610.9 


31.2 


1.3 


90 


2133.6 






2091.1 


0^ 


-42.5 
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TABLE 3 (CON'T) 



REFRACTION COMPARISONS 
FOR THE 

THREE REFRACTION MODELS 



(Continued) 

Coast 



Zenith 
Distance 
(doftroos ) 


and 

Good otic 
Survey 


Baldini 


Garfinkol 


Difference from Coast 
and Geodetic Survoy Model 
Baldini Garfinkel 


Temperature 


■ 35° C 


Pressure 


711 


mm Refraction in seconds 


0 


0.0 


0.0 




0.0 


0.0 


0.0 


5 


4.4 


4.3 




4.4 


0.1 


0.0 


10 


8.8 


8.8 




8.8 


0.0 


0.0 


15 


13.3 


13.3 




13.4 


0.0 


0.1 


20 


18.1 


18.1 




18.2 


0.0 


0.1 


25 


23.2 


23.1 




23.3 


-0.1 


0.1 


30 


28.7 


28.7 




28.9 


0.0 


0.2 


35 


34.8 


34.8 




35.0 


0.2 


0.2 


40 


41.8 


41.7 




41.9 


-0.1 


0.1 


45 


49.8 


49.6 




49.9 


• -0.2 


0.1 


50 


59.3 


59.1 




59.5 


-0.2 


0.2 


55 


71.0 


70.8 




71.2 


-0.2 


0.2 


60 


86.0 


85.7 




86.3 


-0.3 


0.3 


65 


106.2 


105.9 




106.6 


-0.3 


0.4 


70 


135.7 


135.2 




136.1 


-0.2 


0.4 


75 


183.0 


182.3 




183.4 


-0.7 


0.4 


80 


272.9 


271.8 




272.9 


-1.1 


0.0 


85 


506.1 


529.6 




501.3 


23.5 


-4.8 


90 


1771.5 






1632.5 


oO 


-139.0 


Temporature 


35 c C 


Pressure 


737 mm Refraction in seconds 


0 


0.0 


0.0 




0.0 


0.0 


0.0 


5 


*.5 


4.5 




4.5 


0.0 


0.0 


10 


9.1 


9.1 




9.1 


0.0 . 


0.0 


15 


13.8 


13.8 




13.9 


0.0 


0.1 


20 


18.8 


18.7 




18.9 


-0.1 


0.1 


25 


24.1 


24.0 




24.2 


-0.1 


0.1 


30 


29.8 


29.7 




29.9 


-0.1 


0.1 


35 


36.1 


36.1 




36.3 


0.0 


0.1 


40 


43.3 


43.1 




43.5 


-0.2 


0.2 


45 


51.6 


51.5 




51.8 


-0.1 


0.2 


50 


61.4 


61.3 




61.7 


-0.1 


0.4 


55 


73.5 


73.4 




73.8 


-0.1 


0.3 


60 


89.1 


88.9 




89.4 


-0.2 


0.3 


65 


110.1 


109.8 




110.5 


-0.3 


0.4 


70 


140.7 


140.1 




141.1 


-0.5 


0.4 


75 


189.7 


189.0 




190.1 


-0.7 


0.4 


80 


282.9 


281.8 




282.9 


-1.1 


0.0 


85 


524.5 


549.1 




519.8 


24.6 


-4.7 


90 


1835.9 






1695.9 


03 


-140.0 
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TABLE 3 (CON'T) 



REFRACTION COMPARISONS 
FOR THE 



Zenith 

Distance 

(do^roos) 


Coast 

and 

Goodotic 

Survey 


THREE REFRACTION MODELS 
(Continued) 

Difforonco 

: and Goodotic 

Baldi ni Garfinkol Baldl ni 


from C< 
Survey 
Garfl nl 


Temporature 


» 35°c 


Pressure 


760 mm Refraction In coconds 


0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


4.7 


4.6 


4.7 


-0.1 


0.0 


10 


9.4 


9.4 


9.4 


0.0 


0.0 


15 


14.2 


14.2 


14.3 


0.0 


0.1 


20 


19.4 


19.3 


19.5 


-0.1 


0.1 


25 


24.8 


24.8 


24.9 


0.0 


0.1 


30 


30.7 


30.7 


30.8 


0.0 


0.1 


35 


37.2 


37.2 


37.4 


0.0 


0.2 


40 


44.7 


44.5 


44.8 


-0.2 


0.1 


45 


53.2 


53.1 


53.4 


-0.1 


0.2 


50 


63.3 


63.2 


63.6 


—0 • 1 


0.3 


55 


75.8 


75.7 


76.1 


-0.1 


0.4 


60 


91.9 


91.7 


92.2 


-0.2 


0.3 


65 


113.5 


113.3 


114.0 


-0.2 


0.7 


70 


145.0 


144.6 


145.5 


-0.4 


0.5 


75 


195.6 


194.9 


196.1 


-0.7 


0.5 


80 


291.6 


290.6 


291.8 


-1.0 


0.2 


85 


540.7 


566.3 


536.2 


25.6 


-4.5 


90 


1892.7 




1752.2 




-140.5 


Temperature 


35 °C 


Pressure 


787 mm Refraction in seconds 


0 


0.0 


0.0 


0.0 


0.0 


0.0 


5 


4.9 


4.8 


4.8 


-0.1 


- 0.1 


10 


9.7 


9.7 


9.8 


0.0 


0.1 


15 


14.8 


14.7 


14.8 


-0.1 


0.0 


20 


20.1 


20.0 


20.1 


-0.1 


0.0 


25 


25.7 


25.7 


25.8 


0.0 


0.1 


30 


31.8 


31.8 


31.9 


0.0 


0.1 


35 


38.6 


38.5 


38.7 


-0.1 


0.1 


40 


46.3 


46.1 


46.4 


-0.2 


0.3 


45 


55.1 


55.0 


55.3 


-0.1 


0.3 


50 


65.6 


65.5 


65.8 


-0.1 


0.2 


55 


78.5 


78.4 


78.8 


-0.1 


0.4 


60 


95.2 


95.0 


95.5 


-0.2 


0.3 


65 


117.6 


117.3 


118.0 


- 0.3 


0.4 


70 


150.2 


149.8 


150.6 


-0.4 


0.4 


75 


202.6 


201.9 


203.1 


-0.7 


0.5 


80 


302.1 


301.0 


302.2 


-1.1 


0.1 


85 


560.2 


586.5 


555.4 


26.3 


-4.8 


90 


1960.8 




1818.7 




-142.1 



30 



360 

34 0 

320 

300 

280 

260 

240 

220 

200 

180 

160 

140 

120 

100 

80 

60 

40 
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P= 787 Iran 




T>= 10°C 
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'raction Variation for the Coast and Geodetic Survey Model 
>m 0 to 80 Degrees Zenith Distance 
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B= Baldird - Coast Survey 
G= Garfinkel - Coast Survey 
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Baldini - Coast Survey 
Garfinkel - Coast Survey 
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&= Baldini - Coast Survey 
G= Garfinkel - Coast Survey 
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Figure 5. Refraction Differences From Coast Survey Model For High Temperature and Lev Pressure 



CHAPTER 4 



THE LATITUDE REDUCTION METHODS 
4.1 Background on the Reduction of Astronomic Data 

The latitude reductions for this projoct wero done by two compu- 
tation methods which wore usod as a chock on each other for the final 
valuos obtained. The first method employed is that given by Thorson 
in the Second Order Astronomical Position Determination Manual of the 
United States Coast and Geodetic Survoy.(9) This reduction method is 
based in turn on a version given by Chauvenot in A Manual of Spherical 

and Practical Astronomy . Two versions of this method were usod with 

the only difference being the use of higher order terms in the second 

version to attempt to gago their effects on the final position. The 
second reduction method used is based strictly on tho use of spherical 
trigonometry and the astronomical trianglo. A method outlined by 
Chauvenet was used in the final formulation of this method, 

The observed data available for this investigation included 
observations for Polaris and a soloctod south star at each station. 

The maximum zenith distance differences between tho two stars at any one 
station was just less than two degroos. The latitude results wore re- 
duced for both stars and then avoragod to obtain the combined valuo 
for the station. A standard error of tho moan value obtained in aver- 
aging each of these separate latitudes was also computed. 

Astronomic latitude may be defined as the doclination of tho 
zenith of the obsorvor. An alternate way of stating this fact is to say 
that tho latitude is the altitude of tho observer's pole. Both of those 
definitions of latitude will be usod in explaining the reduction 
methods motioned above, 
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Tho reduction methods used in this investing tion aro both based 
on the genoral sphorical trigonometry formulas which aro derived from 
tho astronomical triangle. Thoso gonoral formulas may bo stated as 

sin h = sinySsin^+ cos y‘> cos^ cos n 

cos t = sin h - sin b'sinS 
cos v‘ cos 

with y5 the observer's latitude, h tho altitude, % tho declination 
and t tho hour angle of the star in quostion. Tho differences in the 
two mothods aro in certain approximations that are made and in tho 
final method of use of tho quantities. In the first reduction mothod, 
two separate moans are used to reduce tho north and south stars, while 
in the second reduction mothod, the samo gonoral application is made to 
both stars. ^’9) 

4.2 Reduction Method Basod on the Uso of Polaris and A Circummeridian 
Star 

This method of reduction employs different mothods of reduction 
for the polo star and the south star. Certain approximations are mado 
to reduce the observed position of the star to tho observer's meridian 
and thon employ the definitions given above to determine the latitude 
of the station. 

For the polo star, tho latitude will be treated as the altitude 
of tho polo. For this reduction, tho declination will bo replaced by the 
polar distance which is defined as 90 degrees minus the declination of 
tho star. Substituting this quantity into tho first general equation 
given in Section 4.1 gives the expression 

sin h = sin cos P + cos 0 sin P cos t 
whore P represents tho polar distance. As the value of P is small for 
Polaris (loss than 2 degrees), the latitudo may bo developed in a sories 
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form of asconding powers of P. This serios oxpansion gives the option 
of using as many terms as aro roquired for any desired precision. 
Chauvenet^ 4 ^ obtains this oxpansion by writing tho latitude equation as 

<fi - h - x 

where x is a small correction on the order of magnitude of P. Tho 
series expansion for x is givon as 

x = P cos t + P^ sin^ t tan h - l/3 P^ cos t sin^ t. 

This expansion is accurate to 0.01 seconds of arc. After substitution, 
the final latitude expression becomes 

<fi - h - P cos t + I P^ sin^ t tan h - l/3 cos t sin^ t. 

A factor may be introduced into this equation to convort all values to 
the same units. This would be in tho last two terms where a value of 
sine of one second of arc, to the appropriate power, is introduced as 
a conversion factor if P is in soconds. Thorson does not use the last 
term (P^) in his method of reduction. This limits the mothod to an 
accuracy for the final astronomic latitude of somewhere within ono 
second of arc. (2,9) 

To reduco the circummeridian star, the latitude will bo obtained 
from the declination of tho observer's zenith. Sevoral observations 
of the star may bo made close to the meridian and then reduced to tho 
meridian. These altitudes observed near tho meridian give noarly as 
accurate a result when roduced to tho meridian as if tho observation 
were taken on tho meridian. This accuracy is within one second of arc 
for tho final latitude if the observations aro within 30 minutes of 
timo from tho moridian and within 0.1 seconds of arc if tho observations 
aro within 15 minutes of timo from the moridian. 

From the first general equation of Soction 4,1, Chauvenot^^ 
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develops an expression for tho zonith distance of the observed star. 
Using the trigonometric relation for the hour angle that 

p . 

cos t = 1 - 2 sin 2 t, 
the zenith distance may be obtained from 

cos z = sin h + cos vcos > (2 sin^ j t) 
where z is the observed zonith distance corrected for refraction. / is 
introduced into the equation from the relation that on the meridian 

H = / - p 

for a star south of the observer's zenith. Using tho relationship that 
tho altitude is the complement of the zenith distance, the expression 
may be writ ton as 

2 1 

cos z = cos z. ^ -2 cos f cos b sin 2 t 

whore Z.j is the zenith distance reduced to the meridian. This equation 
is then developed into a power series which in its final form is 
expressed as 



where 



- Am + Bn 



A = cos * cos S 
sin 2 j 

B = A 2 cot Zj 

- . 2 i , 

m = 2 sin 2 t 

4 

n = sin it 

in which m and n may be divided by tho sine of one second to put these 
terms in seconds of arc. For practical computations, the value of 
may be used in place of *2 ^ to determine the valuos of A and B, An 
assumed value of the latitude must also be used for the computa tions 
If a number of observations are takon, a series of these equations are 
obtained and an average value of 2 j will result. The latitude will 
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thon bo obtained from the oxprossion 



which may bo given the alternate form 

</' = S + 90° - hj . 

Using the values determined above, this equation reduces to 

= b + 90° - h + Am - Bn. 

In practice a curvature correction for the time elapsed between 
the direct and reverse instrument sightings is introduced. This is given 
as 

= 0.0001364 (AT 5 ) 2 

where & T 5 is the elapsed timo in seconds between the direct and reverse 
sightings. This correction is added to tho m term of the above ex- 
pression. Ihorson does not use the last term of this expression (Bh) 
in his reduction. The value of the assumed latitude used in this method 
should be as close as possible to the expected final latitude valuo, 

(2,9) 

4.3 A Method for Obtaining the Astronomic latitudes Using Basic 
Spherical Trigonometry 

Ihis reduction method once again makes use of tho first general 
equation given in section 4.1 for the solution to the astronomic 
triangle. The main difference between this method and the one given in 
the preceding section is that no series expansion is used here to obtain 
an expression for the latitude. Instoad, two auxiliary variables, d and 
D, are introduced so that 

d sin D = sin S 
d cos D = cos S cos t. 

For convenience another auxiliary variable, ); , is brought into the 
reduction. This variable is defined by the rolation 
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(j' — D = + r • 

After substitution the general equation may bo expressed as 

d cos (^ - D) = sin h. 

By eliminating d from the final solution, the final computation 
equations are obtained. These equations are given as 

tan D = tan 6 sec t 
cos y = sin h sin D cosec 

/- = D + v 

using the auxiliary variables defined above. The first equation deter- 
mines D which is taken to bo always less than 90 degrees and positivo 
or negative according to tho sign of the tangent. The second equation 
is indeterminate as to the sign of 'ft since the cosine of + i and - $ 
are the same. Therefore, two values of latitude are obtained by the 
third equation; however, one of those values will be over 90 degrees 
and can be discarded. It was found that for Polaris, the D - S' value 
is to be used to obtain tho latitude. For the south star, tho D + £, 
value givos tho desirod result. In theso reductions a curvature 
correction was used for the south star to compensate for the olapsod 
time between tho direct and reverse pointings of the instrument. This 
is of the form Am^ where the quantities are the same as those intro- 
duced in the preceding section. This value was subtracted from that 
value for the south star latitude which resulted from tho D + <5 
procedure. ( 2*9 ) 

4.4 The Station Reductions to Obtain the Astronomic Latitude 

Each station was reduced by threo mothods using each refraction 
model, Tho first reduction used the procedures given by Ihorson, (9) 
The second reduction used this same method but added the higher order 



40 



terms described in section 4.2 The third roduction employed the method 
outlined in section 4.3 All of those reductions wore dono by making 
use of a computor program which used the subroutines for each rofrac- 
tion model. The Aberdoen subroutine was used here for tho Garfinkol 
Model. Those reduction results are given for each station in Table 4. 

As a chock on tho accuracy of the roduction program, tho first set of 
reduction results for the Coast and Geodetic Survey Model was compared 
to hand computed results and found to be in extremely close agreement. 
Tho slight differences, of a maximum value of 0.02 seconds of arc, may 
be attributed partially to the fact that the hand computations used 
averaged values of the observed altitudes to determine the trigonome- 
tric functions used in the reduction. In the program the function was 
computed directly for each observed altitude (corrected for the refrac- 
tion). The agreement bo tween tho final values was regarded as being 
within the accuracy limits required for this investigation. 

In goneral, the final latitude values for tho north and south 
stars, as well as the combined latitude, agreed closely for each of tho 
three programmed mothods employed. The second and third reduction pro- 
grams were generally in close agreement in their results. It was found 
that these two programs reduced the differences between tho computed 
latitudes for the north and south stars by nearly 0.1 second when com- 
pared to the results of the first method. This change may be attrib- 
uted to the greater accuracy of these programs. 

Thorson states that if all field requirements arc met, the final 
computations will show probable errors of loss than £ 0.4 seconds of 
arc. This would be equivelent to a standard error of about * 0.6 
seconds of arc. All of the observations used in this projoct wore with- 
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in this range. The standard errors for the mean values of the latitudes 

determined by oach reduction method are given in Tablo 4. 

4.5 The Effects of Refraction Model Errors Upon the Computed 
Latitude Values. 

The latitude may be computed from the observed zenith distance 
of a star using the definitions given in soction 4.1 For the case of a 
star observed in the meridian, the latitude could bo obtained from the 
equations 

p = (90°-^) + (90° - z) 

or 

0 ^ 5b -Z 

for a star observed north of the observer’s meridian and 

f = >> + z 

for a star south of the observer's meridian. In these equations /> is 
the latitude, S the declination and Z the true zenith distance. In the 
case of a circumpolar star observed at lower transit, the declination 
would have to be replaced by its supplement in tho equations for the 
north star. The true zonith distance is obtained by adding the re- 
fraction to the observed zenith distance. This relationship is expressed 

by 

Z = Z Q + R 

whore Z is the observed zonith distance and R is the refraction corroc- 
o 

tion. Substituting this expression into the latitude equations, tho 
relationships are given as 

f = (90° - b) + (90° - Z q - R) 
or 

t = ib - z„ - R 

for the north star and 
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v A = \+ Z + R 

o 

for the south star. 

It can be seen from the above expressions that the refraction 
term is negative in them to obtain the latitude from the north star and 
positive when latitude is obtained from the south star. Since the signs 
of the refraction terms are opposite for the north and south star lati- 
tude reductions, any constant error in the refraction model from which 
this term was obtained would cause an opposite effect on the latitudes 
computed from the two stars. If the refraction model gives a value 
for the refraction correction which is lower than it should be for all 
zenith distances, the latitude value computed for the north star will 
be greater than it actually is, and the latitude value from the south 
star will be less than it actually is. The converse case is true if 
the refraction model gives refraction values that are consistently high- 
er than they should be. 

In the observations which were used for this investigation, the 
recommended procedure had been followed in the selection of the stars. 
This procedure calls for selecting the north and south stars at 
nearly equal zenith distances. When this is done, since we have seen 
that the refraction gives an opposite effect for the two stars, the 
average value of the two computed latitudes should minimize any errors 
due the refraction model. Since Polaris was used as the north star in 
these observations, south stars with a zenith distance near 50 degrees 
were selected. 

Even when this averaging of the two latitude values is done to 

minimize the refraction errors, it must be pointed out that there will 
still be errors present in this final latitude value due to observation 
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and instrument conditions, but thoso should be smaller and in a random 
distribution. It should bo noted that some refraction model orror will 
still be present after the latitudes are avoragod duo to the fact that 
the stars are not at exactly the same zenith distances and possibly due 
to atmospheric conditions at the timo of tho observation. 

Generally, then, it will be stated that the best refraction 
modol would give the minimum difference between the values of latitudo 
computed from observations of the north and south stars at any given 
time. 

4.6 Evaluation of the Results Obtained for Latitudo from the 
Observation Data. 

It was found that tho standard error of tho moan value of the 
computed latitude was consistent for the throe refraction modols in- 
vestigated, This is true for tho latitudo values computed for the north 
and south stars as well as tho combined latitude. No significant 
chango occurs in apy of theso standard error values due to tho refrac- 
tion model used and, in fact, vory little change occurs when no refrac- 
tion model at all is used. Those values can bo found in Table 4. 

The combined latitudes for each refraction modol wero found to 
be in very close agreement for each reduction mothod employod, with no 
significant difference noted for second order results. The widest 
variation among theso values is in tho range of 0.02 seconds. Some 
variation was found in the computod values for the latitudes that woro 
roduced from each of the two stars. In this light, it was found that 
tho Coast and Geodetic Survey Model gave the least difference between 
the north and south star values of latitude. The Baldini Model gener- 
ally gave just a slight bit more spread with tho Garfinkel Model 
giving the widest range betwoon these two valuos. This difference in 



spread was found to closely equal tho refraction differences. In all 
cases, tho difference between the combined latitude value and those of 
the north and south stars was too great to justify using tho observation 
or just one star to determine the astronomic latitude of a station. 

This is using the criteria stated in Section 4.4 for the accuracy of 
tho observations. 

The reductions were done for both the Aberdeen set of constants 
and the original ones suggested by Garfinkel when the Garfinkel Model 
was used. It was found that the Aberdeen values gave about one second 
less in the difference between tho north star and south star latitude 
values for each station. This was true for all of the stations reduced. 
The Coast and Geodetic Survey Garfinkel program was used for the reduc- 
tions also, and it was found that slightly less difference was obtained 
between tho north and south star latitudes than when the Aberdeen pro- 
gram and values were used. This difference was not significant for 
the purposes of this investigation, having a magnitude of about 0.03 
seconds of arc. 

It sould be noted that tho accuracy of the combined latitude 
would be affcctod to a groater degree by the refraction model as the 
difference in the zenith distances of the observod stars increased. 

At largo zenith distances, the refraction modol would also have more 
effect on tho standard errors of the mean values obtained since tho re- 
fraction becomes much greater as the zenith distance increases. 

From tho results given in Table 4, it can bo stated that for 
second order latitudes observed using a north star and a south star 
with nearly equal zenith distances, any of the three refraction models 
investigated would give accoptablo rosults. 
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TABLE 4 



Refraction 

Model 



Station: 

Temperature: 

Pressure: 

Latitude 
North Starr 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error (+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Combined 
Program 1 
Std Error (+) 
Program 2 
Std Error(+) 
Program 3 
Std Error (+) 
Difference 
N and S 
Latitudes 
Program 1 
Pro graun 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 



Coast and 
Geodetic 

None Survey Baldird Garfinkel 



Carr 
20.88*0 
735.2 mm 


39° 33' 


39° 32 


11 


f1 


18.612 


12.088 


0.361 


0.384 

12.191 

0.385 

12.188 

0.385 


39 31’ 


08.132 


14.092 


0.492 


0.493 

14.092 

0.493 

14.098 

0.493 


39 32’ 


13.372 


13.070 


0.257 


0.257 

13.141 
0.257 

13.142 
0.257 


t! 


ii 


130.480 


2.003 

1.900 

1.910 



39 c 32' 


39 J 32 


ft 


ff 


12.051 


11.824 


0.385 


0.386 


12.154 


11.927 


0.386 


0.386 


12.150 


11.923 


0.386 


0.386 


14.097 


14.322 


0.493 


0.493 


14.098 


14.323 


0.493 


0.493 


14.105 


14.330 


0.492 


0.492 


13.072 


13.072 


0.257 


0.257 


13.125 


13.125 


0.257 


0.257 


13.127 


13.127 


0.257 


0.257 


11 


11 


2.046 


2.498 


1.945 


2.396 


1.955 


2.406 
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TABLE 4 (CON'T) 



Refraction 

Model 



Station: 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error (+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 
latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Geodetic 

None Survey Baldini Garfinkel 



Foust 
l 6 . 90 3 C 
733.3 mm 


39" 42' 


39' 41 


it 


tf 


16.067 


09.328 


0.726 


0.719 

09.428 

0.720 

09.423 

0.720 


39 40' 


07.803 


11.469 


0.583 


0.584 

11.472 

0.584 

11.474 

0.584 


39 42' 


11.934 


10.396 


0.424 


0.417 

10.449 

0.417 

10.448 

0.416 


11 


11 


128.264 


2.141 

2.044 

2.052 



39° 41' 


39° 41 


it 


11 


09.275 


09.079 


0.719 


0.719 


09.376 


09.184 


0.719 


0.719 


09.375 


09.177 


0.720 


0.720 


11.553 


11.738 


0.585 


0.584 


11.556 


11.742 


0.585 


0.584 


11.558 


11.774 


0.584 


0.584 


10.415 


10.408 


0.416 


0.416 


10.468 


10.460 


0.416 


0.416 


10.446 


10.461 


0.416 


0.416 


tl 


11 


2.278 


2.659 


2.180 


2.558 


2.184 


2.566 
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TABLE 4 (CCN'T) 



Refraction 

Model 



Station: 
Temperature : 
Pressure: 

Latitude 
North Star 
Program 1 
Std Error( +) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error (+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error (+) 
Program 3 
Std Error(+) 
Difference 
N and S 
latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCATION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Geodetic 

None Survey Baldird Garflnkel 



Haines 
15.88 c C 
740.1 mm 


39° 33' 


39° 32' 


39° 32' 


39° 32 


It 


It 


tt 


tt 


5^.795 


47.310 


47.253 


47.032 


0.374 


0.377 


0.377 


0.377 




47.396 


47.337 


47.116 




0.377 


0.378 


0.378 




47.390 


47.33^ 


47.112 




0.378 


0.378 


0.378 


39° 31' 


45.886 


50.786 


50.841 


51.054 


0.220 


0.220 


0.220 


0.220 




50.793 


50.864 


51.059 




0.220 


0.220 


0.220 




50.789 


50.884 


51.057 




0.222 


0.221 


0.221 


39 32' 


50.340 


49.050 


49.047 


49.042 


0.162 


0.162 


0.162 


0.162 




49.092 


49.092 


49.086 




0.1O1 


u.162 


0.161 




49.090 


49. 088 


49.083 




0.l6l 


0.161 


0.161 


It 


It 


tl 


tt 


128.909 


3.^76 


3.588 


4.022 




3.397 


3.508 


3.943 




3.398 


3.610 


3.945 
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TABLE 4 (CON'T) 



Refraction 

Modol 



Station: 
Temperature: 
Pressure : 

Latitude 
North Star 
Program 1 
Std Error(+) 
Program 2 
Std £rror(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued ) 



Coast and 
Goodetic 

None Survo.y Baldlni Garfinkol 



Head 
15.88°C 
738.9 mm 


39° 32 ' 


39° 30’ 


39 ^ 30' 


39 ^ 30 


ft 


II 


It 


tt 


01.838 


53.382 


53.301 


53.110 


0.304 


0.294 


0.294 


0.295 




5 3.4S4 


53.383 


53.193 




0.294 


0.294 


0.294 




53.459 


53.380 


53.188 




0.294 


0.293 


0.294 


39 29' 


50.054 


55.740 


55.809 


55.993 


0.347 


0.347 


0.346 


0.347 




55.747 


55.812 


55.998 




0.347 


0.346 


0.347 




55.745 


55.812 


55.996 




0.346 


0.346 


0.346 


39 30 * 


55.944 


54.561 


54.554 


54.551 


0.275 


0.266 


0.265 


0.265 




54.604 


54.599 


54.595 




0.265 


0.264 


0.264 




54.604 


54.595 


54.592 




0.265 


0.265 


0.265 


II 


II 


It 


It 


131.785 


2.358 


2. 508 


2.882 




2.283 


2.429 


2.804 




2.287 


2.432 


2.808 
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TABLE 4 (CON'T) 



Refraction 

Model 



Station: 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Geodetic 

None Survey Bald ini Garfinkel 



Hoyt 
19.21 C 
738.6 mm 


39° 40' 


39' 39 


11 


If 


26.719 


19.949 


0.420 


0.456 

20.047 

0.452 

20.045 

0.453 


39" 38' 


18.996 


22.588 


0.382 

r\ 


0.378 

22.589 

0.378 

22.589 

0.378 


39 39' 


22.857 


21.266 


0.339 


0.347 

21.319 

0.347 

21.317 

0.347 


t! 


11 


127.722 


2.639 

2.542 

2.544 



39- 39, 


39° 39 


11 


11 


19.953 


19.740 


0.455 


0.455 


20.052 


19.839 


0.454 


0.454 


20.049 


19.838 


0.454 


0.455 


22.618 


22.818 


0.378 


0.378 


22.622 


22.818 


0.378 


0.378 


22.622 


22.824 


0.379 


0.378 


21.283 


21.279 


0.348 


0.348 


21.336 


21.333 


0.348 


0.348 


21. 33^ 


21.329 


0.348 


0.348 


If 


11 


2.666 


3.078 


2.570 


2.994 


2.573 


2.987 
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TABLE 4 (CON'T) 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Refraction 

Model 


None 


Coast and 

Geodetic 

Survey 


Baldini 


Station: 


Imler 






Temperature: 


20.53"C 






Pressure: 


735.7 mm 






Latitude 


39° 32' 


39^ 31' 


39* 31' 


North Star 


it 


It 


It 


Program 1 


43.329 


36.736 


36.723 


Std Error(+) 


0.295 


0.302 


0.302 


Program 2 




36.834 


36.820 


Std Error(+) 




0.302 


0.302 


Program 3 




36.829 


36.817 


Std Error(+) 




0.302 


0.302 


South Star 


39 " 30 ' 






Program 1 


36.789 


39. 5M 


39.576 


Std Error(+) 


0.187 


0.187 


0.187 


Program 2 




39.5^5 


39.577 


Std Error(+) 




0.187 


0.187 


Program 3 




39.552 


39.584 


Std Error (+) 




0.187 


0.187 


Combined 


39° 31' 






Program 1 


40.058 


38.137 


38.149 


Std Error(+) 


0.187 


0.189 


0.189 


Program 2 




38.190 


38.199 


Std Error(+) 




0.189 


0.189 


Program 3 




38.190 


38.201 


Std Error(+) 




0.189 


0.189 


Difference 








N and S 








Latitudes 


fl 


II 


II 


Program 1 


126.540 


2.805 


2.854 


Program 2 




2.710 


2.756 


Program 3 




2.722 


2.768 



Garfinkel 



39* 31' 

n 

36.498 

0.302 

36.597 

0.302 

36.594 

0.302 

39.785 

0.187 

39.787 

0.187 

39.794 

0.187 

38.142 

0.189 

38.192 

0.189 

38.194 

0.190 



3.288 

3.190 

3.200 
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TABLE 4 (CON'T) 



Refraction 

Model 



Station: 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error(+) 
Program 2 
Std Srror(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error (+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error (+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Geodetic 

None Survey Bald ini Garfinkol 



Lanum 
12.98 C 
733.6 mm 


J)' J 40' 


39 7 39 


ii 


II 


50.997 


43.111 


0.271 


0.256 

43.213 

0.256 

43.208 




0.255 


39 3 8 * 


35.678 


43.575 


0.199 


0.199 

43.578 

0.199 

43.580 

0.200 


39° 39' 


43.388 


43.343 


0.139 


0.131 

43.395 

0.130 

43.393 

0.130 


II 


II 


135.219 


0.464 

0.366 

0.372 



39° 39' 


39° 39 


II 


II 


42.998 


42.825 


0.255 


0.255 


43.101 


42.928 


0.255 


0.255 


43.096 


42.923 


0.255 


0.255 


43.686 


43.862 


0.200 


0.199 


43.690 


43.862 


0.199 


0.199 


43.693 


43.876 


0.199 


0.199 


43.341 


43.343 


0.130 


0.130 


43.395 


43.393 


0.130 


0.130 


43.393 


43.395 


0.130 


0.130 


II 


II 


0.688 


1.036 


0.598 


0.934 


0.597 


0.944 
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TABLE 4 (CON’T) 



Refraction 

Model 



Station: 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Geodetic 

None Survey Baldlni Garfinkel 



Meinfeller 
16.85°C 
737.3 rom 


39° 39’ 


39* 37' 


37’ 


39° 37 


n 


11 


11 


11 


07.178 


59.370 


59.318 


59.119 


0.239 


0.236 


0.236 


0.236 




59.452 


59.399 


59.201 




0.236 


0.236 


0.236 




59.449 


59.394 


59.196 


39° 36' 


0.236 


0.236 


O.236 


39 38’ 


39 38' 


39 38 


55.221 


00.813 


00.877 


01.069 


0.248 


0.245 


0.245 


0.245 




00.818 


00.884 


01.074 




0.245 


0.245 


0.245 




00.818 


00.884 


01.074 




0.245 


0.245 


0.245 


39* 38* 


01.199 


00.092 


00.097 


00.094 




0.159 


0.159 


0.159 




00.137 


00.140 


00.137 




0.159 


0.159 


0.159 




OO.133 


00.140 


00.135 




0.159 


0.159 


0.159 


fl 


11 


It 


11 


131.956 


1.444 


1.558 


1.950 




1.366 


1.485 


1.873 




1.370 


1.490 


1.878 
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TABLE 4 (CON'T) 



Refraction 

Model 



Stati on: 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Krror(+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Geodetic 

None Survey Baldinl Garfinkol 



Owens 
21.69 & C 
736.4 mm 


390 4oi 


39 ° 39 


11 


11 


42.374 


36.151 


0.345 


0.360 

36.254 

0.360 

36.252 

0.360 


39 38' 


33.001 


39.283 


0.526 


0.514 

39.295 

0.509 

39.275 

0.519 


39* 39' 


37.685 


37.716 


0.301 


0.282 

37.774 

0.282 

37.762 

0.285 


11 


11 


129.373 


3.133 

3.042 

3.023 



39* 39’ 


39* 39 


II 


II 


36.216 


35.982 


0.361 


0.361 


36.315 


36.084 


0.361 


0.361 


36.315 


36.082 


0.361 


0.361 


39.222 


39.455 


0.513 


0.514 


39.235 


39.469 


0.509 


0.509 


39.218 


39.448 


0.519 


0.519 


37.718 


37.716 


0.281 


0.281 


37.773 


37.774 


0.281 


0.281 


37.766 


37.766 


0.284 


0.284 


II 


If 


3.006 


3.473 


2.920 


3.385 


2.903 


3.366 
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TABLE 4 (CON'T) 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 



Refraction 

Modol 


None 


Station: 


Persinger 


Temperature: 


19.72°C 


Pressure: 


741,2 mm 


Latitude 


39° 33' 


North Star 


11 


Program 1 


20.578 



Std Error (+) 0.264 

Program 2 
Std Error(+) 

Program 3 
Std Error(+) 



South Star 


39 31 


Program 1 


09.767 


Std Error(+) 
Program 2 
Std Error(+) 
Program 3 


0.567 


Std Error(+) 


O 


Combined 


39 32 


Program 1 


15.171 


Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 


0.314 


Latitudes 


II 


Program 1 
Program 2 
Program 3 


130.811 



Geodetic 

Survey Baldini 



39^ 32* 


39 0 32 


11 


II 


13.098 


13.039 


0.275 


0.276 


13.199 


13.137 


0.276 


0.277 


13.197 


13.135 


0.275 


0.276 


14.775 


14.838 


0.561 


0.561 


14.782 


14.785 


0.559 


0.559 


14.776 


14.842 


0.562 


0.562 


13.937 


13.937 


0.307 


0.307 


13.989 


13.992 


0.307 


0.306 


13.987 


13.989 


0.307 


0.307 


II 


II 


1.777 


1.779 


1.503 


1.708 


1.580 


1.706 



Garfinkel 



39 ° 32 ' 

II 

12.820 

0.276 

12.921 

0.277 

12.917 

0.277 

15.046 

0.561 

15.055 

0.559 

15.051 

0.562 

13.935 

0.307 

13.985 

0.307 

13.985 

0.307 



2.227 

2.133 

2.134 
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TABLE 4 (CON'T) 



Refraction 

Hodol 



Stations 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error(+) 
Program 2 
Std Error (+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Geodetic 

None Survey Baldlni Garflnkel 



Tarbill 
24.01°C 
737.9 mm 


39° 33' 


39° 32* 


39" 32' 


39 0 32 


n 


u 


tl 


tl 


08.683 


02.767 


02.788 


02.532 


0.395 


0.385 


0.385 


0.385 




02.870 


02.887 


02.633 




0.385 


0.385 


0.385 




02.685 


02.884 


02.630 




0.385 


0.385 


0.385 


39 31' 


02. >1 


04.563 


04.570 


04.806 


0.505 


0.510 


0.510 


0.510 




04.570 


04.573 


04.813 




0.511 


0.511 


0.511 




04.571 


04.578 


04.817 




0.509 


0.509 


0.509 


J)° 32' 


05.510 


03.665 


03.677 


03.670 


0.165 


0.154 


0.154 


0.154 




03.720 


03.730 


03.723 




0.155 


0.154 


0.154 




03.720 


03.730 


03.722 




0.155 


0.154 


0.154 


II 


It 


If 


II 


126.343 


1.796 


1.782 


2.274 




1.700 


1.686 


2.180 




1.706 


1.694 


2.187 
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TABLE 4 (CON'T) 



Refraction 

Model 



Station: 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error( + ) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error (+) 
Combined 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCATION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued) 



Coast and 
Good otic 

None Survey Baldird Garfinkel 



Taylor 
21.45 c C 
732.0 mm 


39° 41* 


39 c 40 


ft 


ft 


38.837 


32.646 


0.680 


0.689 

32.730 

0.690 

32.728 

0.690 


39° 39' 


31.123 


35.329 


0.294 


0.291 

35.388 

0.291 

35.334 

0.292 


39 * 40' 


34.977 


33.987 


0.289 


0.296 

34.031 

0.296 

34.031 

0.295 


ft 


f f 


127.715 


2.683 

2.607 

2.606 



39^ 40' 

ft 


39° 40 

ff 


32.639 


32.407 


0.690 


0.690 


32.722 


32.490 


0.691 


0.691 


32.718 


32.490 


0.691 


0.691 


35.355 


35.576 


0.291 


0.291 


35.362 


35.585 


0.291 


0.291 


35.360 


35.583 


0.292 


0.292 


33.995 


33.992 


0.296 


0.296 


34.038 


34.036 


0.297 


0.297 


34.038 


3^.035 


0.296 


0.296 


ff 

2.716 


If 

3.168 


2.641 


3.095 


2.642 


3.093 
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TABLE 4 (CON'T) 



Refraction 
Model 



Station: 

Temperature: 

Pressure: 

Latitude 
North Star 
Program 1 
Std Error (+) 
Program 2 
Std Error(+) 
Program 3 
Std Error( + ) 
South Star 
Program 1 
Std Error(+) 
Program 2 
Std Error(+) 
Program 3 
Std Error(+) 
Combined 
Program 1 
Std Error (+) 
Program 2 
Std £rror( + ) 
Program 3 
Std Error(+) 
Difference 
N and S 
Latitudes 
Program 1 
Program 2 
Program 3 



LATITUDE REDUCTION RESULTS 
USING THE 

THREE REFRACTION MODELS 
(Continued ) 



Coast and 
Geodetic 

None Survey Baldinl Garfinkel 



Williams 
l6.75°C 
730 .4 aim 


39° 33' 


39° 32 


II 


II 


33.210 


25.497 


0.367 


0.370 

25.524 

0.370 

25.519 

0.370 


39" 31' 


26.538 


28.509 


0.607 


0.599 

28.513 

0.597 

28.515 

0.601 


39" 32' 


29.874 


27.002 


0.343 


0.336 

27.019 

0.334 

27.016 




0.335 


H 


II 


126.672 


3.012 

2.988 

2.996 



39° 32' 


39° 32 


It 


II 


25.377 


25.179 


0.370 


0.370 


25.404 


25.207 


0.370 


0.370 


25.401 


25.203 


0.371 


0.371 


28.619 


28.800 


0.599 


0.599 


28.626 


28.806 


0.597 


0.597 


28.626 


28.805 


0.601 


0.601 


26.999 


26.987 


0.336 


0.335 


27.016 


27.006 


0.334 


0.334 


27.013 


27.004 


0.336 


0.336 


11 


II 


3.242 


3.621 


3.222 


3.600 


3.226 


3.602 
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CHAPTER 5 



A SUGGESTION FOR AN ADJUSTMENT PROCEDURE 
USING A FORM OF THE GARFINKEL MODEL 
5.1 Background for the Adjustment Method 

Since the atmosphere and its refraction effects can be varied 
by the conditions which provail concerning woathor, season or turbu- 
lance at any given time, there will be some factors which can not be 
accounted for by the assumptions for which the refraction model has 
been constructed. These effects will not be exceptionally large, but 
their presence can effect the results obtained for the refraction at a 
station. There also may be some errors in the refraction model itself 
that can be minimized by the use of an adjustment procedure. It might 
then be possiblo to construct an adjustment program which would perform 
an adjustment on certain of the refraction coefficients in order to give 
refraction results which will reflect the conditions at the station 
where the observations were taken. If many stations wero treated in 
this manner, then the coefficients used could be examined for any general 
trends exhibited. This could lead to revisions and a more accurate re- 
fraction model. 

To truly reflect tho differences in the refraction coefficients 
and to take into account the effect of any observation errors, the 
General Mothod of Adjustments^ 10 ^ should be used for this typo of pro- 
ject. This method considers both the variation of assumed parametors 
and the effect of the observations. This system was used in an in- 
vestigation conducted to determine the feasibility of this suggested 
type of station adjustment. 

In the investigation conducted, two variations of the samo 
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basic form were employed. Tho first form used the condition that the 
values of latitude computed from the observation of the north and south 
stars should bo tho same. The socond form is more general and assumes 
that the first general equation derived from spherical trigonometry and 
the astronomical triangle, given in Section 4.1, should be satisfied 
for the observed star after correction for tho refraction. These two 
methods will be discussed individually in tho sections to follow. 

5.2 The Form of the Garfinkel Refraction Model Selocted 

The form of tho Garfinkel Refraction model suggested by ACIC 
was chosen for this adjustmont procedure. This form was discussed 
in Section The refraction is thon obtained from the relation that 

RaW \T~To (N^ tan & + Ng tan tan-*©) 

where 

W = P/ 2 
r/ Tq 

tan e> = To Tan Z 

* 

and the value of is chosen as 8.7137. All of these quantities have 
been previously defined. For tho purposes of an adjustment, it was 
considered that the pressure, temperature and gamma factor wore con- 
stant. The gamma factor used here is not tho one suggested in the ACIC 
article but is that of the Aberdeen and Coast and Geodetic Survey 
Garfinkel programs which was found to givo less difference in tho com- 
puted latitudes of the north and south stars. Sinco tan Z Q is the samo 
as cot h Q , tho observed altitudo was substituted into the adjustment 
equations for the obsorved zenith distance. The assumed parameters 
become Kj , N 2 and from this oquation. These factors were assumed 

from the refraction results given by the Garfinkel Model for zenith 
distances in tho 40 to 60 dogroe rango, with an approximate moan 
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value taken for each factor. These assumed values arc given in 

Appendix II along with an explanation of the adjustment procedures. 

5*3 Adjustment Method Using a Mathematical Structure Dealing with 
the Latitudes Computed from the North and South Stars 

In this method the mathematical structure is 

=0 

where the subscripts denote from which observod star the latitude 
was computed. The declination is considered as a fixed quantity in 
this adjustment. The observod quantities aro considered to be the hour 
angle and altitude of both the north and south stars. This results in 
a B matrix with dimensions of 4 by 8 which consists of the partial 
derivatives of the mathematical structure with respect to each of the 
observed quantities. The weight of all of the observations has been 
considered to be equal. The parameters are considered to bo the three 
factors from the refraction equation and the assumod latitudo used for 
south star computations. This gives an A matrix of dimensions 4 by 8 
which consists of tho partial derivatives of the mathematical structure 
with rospect to the parameters. No weights were assigned tho parameters 
for this adjustment. Tho basic formulas usod to obtain the corrections 
to the parameters are given as 

AX + BV + W = 0 
X = -(A*M" 1 A) _1 A'MW 
K = BB* 

where the X matrix gives the corrections to the assumed parameters. 

The W matrix is computed for each observation and each element is the 
number which would have to be added to the equation, computed from the 
observed values and assumed parameters, to fulfill the condition given 
by the mathematical structure. The components for these matrices and 
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and some additional information on this adjustment method aro given in 
Appendix II. 

5.4 Adjustment Kothod with a Koro General Mathematical Structure. 

The second method used the same computed quantities but a 
different mathematical structure that gives a condition which oach ob- 
served star must fulfill. This mothod has the same parameters and fixed 
values as the previous method but has a 3 matrix with the dimensions 2 
by 16 , since there are only two observod quantities for each observation. 
These are the hour angle and the altitude of each observed star. The 
same final adjustment matrix equations are used for this form as for 
the previous one. This is a more general form and if it wore found 
workable would give more room for variation in the stars observed as 
two south stars could bo substituted for the north and south star of 
the previous mothod. The same thing could bo done using two north 
stars also. This method could be expanded to use more data than that 
provided by just two stars because of its general form. 

The investigation for this model usod only two stars duo to the 
lack of any other available data. The matrix components used and other 
information on tho adjustment aro given in Appendix II. 

5.5 Results of the Adjustment Method Investigation 

Tho results obtained from tho investigation of those two suggest- 
ed adjustment mothods aro not in any way conclusive in a positive sonso. 
It was found that the obsorvod data were too strongly correlated to 
give any useable results. This was caused by a vory unstable M matrix 
which in turn caused tho inverse K matrix to bo non-symmetrical and 
thcreforo caused tho adjustment results to be unuscablo. This corre- 
lation may bo due to the fact that all of tho observod altitudes 
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were within two dogrees of each other. 

Therefore, nothing can bo said in tho way of constructive con- 
clusions from the adjustment results except that they do not work for 
tho data availablo for this project. A suggested positive test for 
these methods would bo to obsorve stars, both north and south of tho 
zonith, at different zenith distances and hour angles and then uso tho 
adjustment procoduros. The reduction procedures for theso observations 
would not be the same as those used in this project and now equations 
would have to bo written wd.th the mathematical structuro given in 
Section 5»3t as tho forms given in Appendix II are for tho case of Polaris 
and a circummoridian star. A difference in zenith distancos of from 
20 to 30 degrees is recommonded. These zenith distances should be ob- 
served up to about 70 degrees for the study. 

If this type of station adjustment for the refraction coeffi- 
cients was found feasible, then a sequence of observations, say on two 
stars, could bo taken before and aftor the latitude observations at tho 
station. It would be bost if all observations woro taken at a single 
station over a long period of timo to doterraino any consistont differ- 
ences in the refraction model. Howovcr, data taken at difforont 
stations could bo usod in tho project for comparison purposes. 

If it wero provod to bo feasible, this typo of station adjust- 
ment might eventually load to a situation whore the latitude of a 
station could be determined, for socond order accuracy, with the obser- 
vation of only one star. The station adjustment could also be employed 
as a tool for determining seasonable variations and thoir effocts on 
the refraction model. Thoroforo, tho idoa doos havo some merit and 

should be investigated rather throughly before being completely 
discarded. 
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CHAPTER 6 



SUMMARY ALT) CONCLUSIONS 

The basic objective of this project was to determine if the 
refraction model used in second order astronomic latitude determinations 
had apy effect on the final latitude results. The method of observing 
both a north and south star at nearly the same zenith distance was used. 
It can be stated, based upon the results of this project, that the final 
combined astronomic latitude is not significantly affected by the re- 
fraction model used. This is in reference to the three refraction 
models used in this investigation. It may also be noted that the stand- 
ard error of the mean for the computed latitudes is not affected to apy 
significant degree by the refraction model used in reductions. 

It was found that with the three refraction models used, that it 
would not be feasible to observe just one star, north or south, with 
an expectation of obtaining results of acceptable second order accuracy. 
This is based on the consideration that a standard error of i 0.6. 
seconds of arc would be the acceptable criteria. The spread given be- 
tween the latitude values computed for the north and south stars was 
greater than this value in each station computed and for each refraction 
model. 

If the difference or spread obtained between the north and south 
star latitudes can be used as a criteria, it can be seen that the 
Garfinkel Model gives a consistently larger difference, at the zenith 
distances, observed (about 50 degrees), than either the Coast and 
Geodetic Survey Model or the Baldini Model. These latter two models 

give results very close to each other in all the stations computed. 

In a comparison of the refraction models at various temperatures 
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and pressures, which ranged to extremes of each quantity. It was found 
that the Coast and Geodetic Survey and the Baldini Models are consist- 
ently closer to each other in the refraction for a specified zenith 
distance, except for zenith distances over 30 degrees, than the results 
obtained from the Garfinkol Model. Since the Baldini Model was only 
dosignod for zenith distances up to 75 dogroes, this sooms to be a good 
comparison. It must be noted, however, that up to a zenith distance of 
50 degrees, the difference in the refractions given by the three models 
is less than 0.3 seconds of arc. The Garfinkol Model is the only one of 
the three that were investigated which is designed to give refraction 
for zenith distances of 90 degrees or larger. 

The Baldini Model was investigated for the effects of relative 
humidity and the wave length of the observed light. It was found that 
the humidity did not have a very significant effect on the refraction 
output of the model until a zenith distance of about 70 degrees was 
reached. The difference in wave lengths of the received light causes 
a greater variation, but its effect is diminished when it is considered 
that nearly all the observable stars give off light in the yellow wave 
length region of the spectrum. Consequently , the Baldini model was 
used with a relative humidity of 60 percent and a wave length of yel- 
low light equivalent to that used for the basic theory of the Coast 
and Geodetic Survey Model. The results for the Baldini Model would 
have been a bit more accurate if the relative humidity had been taken 
at the observing stations and the wave length of the observed light 
known. It is considered that, for the purposes of this project, these 
factors would not have had a very great influence. 

It was attempted to ascertain if an adjustment for the con- 



ditions at the observing station could be made to improve the refrac- 
tion model. Sufficient data were not available to determine if this 
adjustment was feasible. It is suggested that this area could be one 
of future research but that data should be taken at a variety of zenith 
distances to determine if the adjustment method would actually improve 
the refraction results. 

Finally, it may bo said that the three refraction models in- 
vestigated in this project are adequate for second order latitude de- 
termination employing the present two star observation methods; how- 
ever, it must be noted that none of these refraction models gives the 
ultimate solution to the refraction problem and, therefore, this area 
still remains open to future investigation. 



APPENDIX 
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REFRACTION PROGRAM INFORMATION 



In all of the refraction subprograms used in this investigation 
and listed in this Appendix, the input and output values aro in the samo 
units. For the calling parameters the observed zenith distance is in 
radians, the tempera ture is in degroes contigrado, tho prossure is in 
inches of mercury and the relative humidity is given as a decimal frac- 
tion. All the subroutines give the output for refraction in seconds of 
arc. The requirements of each specific subroutine will now be listed. 

The Coast and Geodetic Survey Subroutine requires that the 
values of refraction from Table V of U. S. C.& G S Special Publication 

(3) 

237 be read into the main program. Since these are listed for 
every ten minutes of arc this consists of reading in 5^1 values. At 
the same time a corresponding quantity in degrees is generated by the 
program. The Subroutine is called with the calling parameters in the 
following order: Relative Humidity, Temperature, Pressure, Zenith 

Distance. Literals statements are included in the subprogram to de- 
termine the factors for temperature and pressure conditions due to 
deviation from the standard conditions assumed for the computation of 
the tables. These values are from Tables VI and VII of tho publication 
mentioned above. 

Tho Baldini Subroutine uses a literals statement to includo a 
table from which the water vapor pressure at the observing station may 
be computed. This is Table I of the Baldini article. The value 
"Slite" in the program is the wave length of light used in the computa- 
tions. This subprogram is called in the following order: Relative 

Humidity, Temperature, Pressure, Zenith Distance. 
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All of tho Garfinkel Subroutines listed in this Appondix have 
the same calling parameters. Those are called in tho following order: 
Zenith Distance, Pressure, Temperature. In all of the programs the 
gamma factor and other constants used are included through tho use of a 
literals statement. 

The program for a comparison of the refraction models is fairly 
straightforward. It reads in the values required for the Coast and 
Geodetic Survey Subroutine and then proceeds to call each of the sub- 
routines given above to determine the refraction for various temperature 
and pressure combinations. Since these combinations range to oxtremos, 
the Coast and Geodetic Survey Subroutine had to be extended to include 
temperature and pressure correction factors for the entire range. This 
was done by simply expanding the literals statement for these factors 
to cover more range of the respective tables they were taken from. 

This expanded form is not shown here. 

The last item in this Appendix is a comparison of the various 
versions of the Garfinkel Subprogram. The Aberdeen and Coast and 
Geodetic Survey comparisons were made for the newer constants and a 
gamma factor of 8 . 7137 . Tho other two comparisons used Garfinkel' s 
original constants and a gamma factor of 8.1578. 



PROGRAM STATEMENTS AND SCATRAN SUBROUTINE 
FOR THE 

U. S. COAST AND GEODETIC SURVEY REFRACTION MODEL 



DIMENSION (RM(541),ZA(541))- 
START READ IMPUT, USREF, ((RM(K) ,K=O t l ,K.LE540))- 
F USREF (12F5.1)- 

DO THROUGH (TABLE), 1=0, 1 .I.L.540- 
DO THROUGH ( TABLE) , J=0, 1 , J. L.6- 
IJ = I + J- 

ZA(IJ) = (1/6.0) + (J * 0.1666666667)- 
TABLE C0N7TNUE- 

ZA(540) = 90.0- 

SUBROUTINE ( RCGS) = REFCGS . (HUM, TEMP, PRESS, ZRS)- 
DIMENSION (CENT(46),C1M(46),BAR(30),CEM(30))- 
UIIIVERSAL (RM.ZA)- 
ZDR = ZRS * 57. 2957797- 

LITERALS (CENT, 5.0, 5.6,6. 1,6. 7. 7.2, 7.8, 8 . 3 . 8.9,9.4.10.0, 

10.6.11.1.11.7.12.2.12.8.13.3.13.9.14.4.15.0. 15.6.16.1 , 

16.7.17.2.17.8.18.3.18.9.19.4.20.0. 20.6.21.1.21.7.22.2 

22.8.23.3.23.9.24.4.25.0. 25.6.26.1.26.7.27.2.27.8.28.3, 
28.9,29.4,30.0)- 

LITERALS (CTM, 1.018, 1.016, 1.014, 1.012, 1.010, 1.008, 1.006, 

1.004.1.002.1.000. 0.998.0.996.0.994.0.992.0.990.0.988, 
0.986,0.985,0.983,0.981,0.979,0.977,0.975,0.973,0.972, 

0.970,0.968,0.966,0.964,0.962,0.961,0.959,0.957,0.955, 

0.953,0.952,0.950,0.948,0.946,0.945,0.943,0.941,0.939, 

0.938,0.936,0.934)- 

LITERALS (BAR, 28. 1,28. 2, 28. 3 , 28. 4, 28. 5, 28. 6 , 28. 7, 28. 8 , 

28.9.29.0. 29.1.29.2.29.3.29.4.29.5.29.6.29.7.29.8.29.9, 

30.0. 30.1.30.2.30.3.30.4.30.5.30.6.30.7.30.8.30.9.31.0,- 

LITERALS (cm, 0.939, 0.942, 0.946, 0.949, 0.953, 0.956, 0.959, 
0.963,0.966,0.970,0.973,0.976,0.979,0.983,0.986,0.989, 
0.992,0.996,0.999,1.003,1.007,1.010,1.013,1.016,1.020, 
1.023,1.026,1.029,1.033,1.036)- 



CT = 0- 



D0 THROUGH (GEDT), IG=0, 1 ,IG. L.^5- 
JG = IG + 1- 

PROVI DED ( TEMP . GE . CENT( I G ) . AND . TEMP . LE . CENT( JG ) ) , 

CT = CTM(IG) + (CTK(JG) - CTM(IG)) * (TEMP - CENT(IG) )/ 
((CENT(JG) - CENT(IG) )- 

PROVIDED (CT.G.O), TRANSFER (G01 )- 

GEDT CONTINUEJ- 

G01 CB = 0- 

DO THROUGH ( GEDP ) , IB=0,1 .IB.L.29- 
JB = IB + 1- 

PROVIDED (PRESS. GE.BAR(IB). AND. PRESS. LE.BAR(JB)), 

CB = CB-I(IB) + (CEM(JB) - CEK(IB) ) + (PRESS - 
BAR(IB))/(BAR(JB) - BAR(IB))- 

PROVIDED (CB.G.O. ) , TRANSFER (G02)- 

GEDP CONTI NUE- 

G02 RT = 0- 

DO THROUGH (GETZ), 1=0, 1,1 .L.5^0- 
J = I + 1- 

PROVIDED (ZDR.GE.ZA(I).AND.ZDR.LE.ZA(J)), RT = RN(I) + 
(RM(J) - RM(I)) * (ZDR - ZA(l))/(ZA( J) - ZA(I))- 

PROVIDED (RT.G.O. ) , TRANSFER (FINAL)- 

GEDZ CONTI NUE- 

FINAL RCGS = RT * CT * CB- 

NORMAL EXIT- 

END SUBPROGRAM- 

END PROGRAM (START)- 



SCATRAN SUBROUTINE 
FOR THE 

BALDINI REFRACTION MODEL 



SUBROUTINE (RB) = REFBAL. (RH,TDC,ATPRS,ZST)- 
DIMENSION ( HEAT( 1 0 ) , HG( 1 0 ) ) - 
PATH = ATPRS * 25. 39998- 

LITERALS (HEAT, -5. 0,0. 0,5. 0,10. 0,15. 0,20. 0,25. 0,30.0, 
35.0,40.0)- 

UTERALS (HG, 3 . 02, 4.58, 6 . 54, 9. 21, 12. 79, 17. 55, 23. 78, 
31.86,42.23,55.40)- 

HGMM = 0 - 

DO THROUGH (VAPOR), IF = 0.1.IF.L.9- 
JF = IF + 1- 

PROVIDED (TDC.GE.HEAT(IF) .AND.TDC.LE.HEAT( JF) ), HGMM= 
HG(IF) +( HG( JF ) - HG(IF)) * ( TDC - HEAT(IF))/5.0- 

PROVIDED (HGMM.G.O) , TRANSFER (CONT)- 

VAPOR CONTINUE - 

CONT E = RH * HGMM- 

SLITE = 0.578- 

AO = 0.99827- 

A1 = -0.00130- 
A2 = 0.000006 

FNG » 2876.04 = 16.288/(SIITE.P.2) + 0. 1 36?(SLITE.P.4)- 
FNG = FNG * 1.0.X.-7- 

FNO = (FNG/ ( 1.0 + 0.00367 * TDC)) * PTAM/ 76 O.O - 
( 5 . 5 .X. - 8 ) * E/( 1.0 + 0.00367 * TDC)- 

FNO = FNO * 206264.81- 

TAN = SIN.(ZST)/COS.(ZST)- 

RB = FNO * AO * TAN + FNO * A1 * (TAN. P.3) + FNO * A2 * 
(TAN.P.5)- 
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NORMAL EXIT- 



END SUBPROGRAM- 



SCATRAN SUBROUTINE 
FOR THE 

GARFINKEL REFRACTION MODEL 
ABERDEEN VERSION 

SUBROUTINE (R) = REFGAR. (ZOB.PP.TN)- 
CALL SUBROUTINE ( ) = FPSET. (3. HALT) - 
HALT CONTI NUE- 

UTERALS ( FN, 4.256, FK, 4472. 8, BE, 0.044385, GAMA, 8. 7137)- 

UPPER COMMON (IND)- 

BOOLEAN (IND)- 

DIMENSION (B(5))- 

PR = PP/ 760.0 * 25.3998- 

T = m/273.16 + 1.0- 

W = pr/t/t- 

TRANSFER (GR15.GR12.GR12) PROVIDED (ZOB - 0.7853981634)- 
GR15 TANTA = SIN. ( ZOB)/ COS. ( ZOB) ) * SQRT. ( T)/GAMA- 
THETA = ATAN.( TANTA )- 
TRANSFER (GR120)- 

GR12 TANTA = GAMA * (COS .( ZOB)/SIN. ( ZOB) )/SQRT. ( T)- 
THETA = ATAN. (1.0/ TANTA )- 
PROVIDED ( TANTA. E.O), THETA = ZOB- 
GR120 PS = 0.0- 
S = 0.0- 
SUK =0.0- 

DO THROUGH (GR33). 1=1 .1,1 .LE. 5- 
DO THROUGH (GR32), J=1 , 1 , J. LE.I- 
FI = I -1- 
FJ = J -1- 

FM = FN * (FI - FJ + 1.0) - FI- 
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FJJ = FJ + 1 .0- 



CALL SUBROUTINE ( ) = GAMMA. (FIJJ.FIJJ)- 
PROVIDED (IND), TRANSFER (GR200)- 
FIJJ = FI - FJ + 1.0- 

CALL SUBROUTINE ( ) = GAMMA. (FIJJ.FIJJ)- 
PRO VIDEO (IND), TRANSFER (GR200)- 
FMS = FM + 1.0- 

CALL SUBROUTINE ( ) = GAMMA. (FM,FMG)- 
PROVIDED (IND), TRANSFER (GR200)- 
CALL SUBROUTINE ( ) = GAMMA. (FMS,FMS )- 
PROVIDED (IND), TRANSFER (GR200)- 
D = FMG * FMS- 

GR100 TRANSFER (GR13.GR31 ,GR13) PROVIDED (FM - S)- 
GR13 D = (FM + S)/(FM -S)*D- 
KK = 2.0 * S + 1.0- 
CH = COS. ( THETA/2. 0).P.KK- 
SH = SIN. (THETA/2.0). P.KK- 
TA = SH/CH- 

TRANSFER (GR132, GR131, GR131) PROVIDED (SH - 0.707106 78 )- 
GR131 CN = CH/SH- 
TA = 1.0/CN- 

GR132 PI = FMG * FMG/D * TA- 

TRANSFER (GR14, GR30, GR14) PROVIDED (FM - S - 1.0)- 
GR14 D = (FM + S + 1.0)/(FM - S - 1.0) * D- 
KK + S * 2.0 + 3.0- 
ST = SIN.(THETE/2.0).P.KK- 
CT = C0S.(THETE/2.0).P.KK- 
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TN = ST/CT- 

TRANSFER (GR142, GR141, GR141 ) PROVIDED (ST - 0.7.710678)- 
GR141 CN = CT/ST- 
TN = 1.0/CN- 

GR142 PI + PI + FMG * FMG/D * TN- 
GR143 PS = PS + PI- 

TRANSFER (GR16, GR31, GR16) PROVIDED (Pl)- 
GR16 S = S + 2.0- 

TRANSFER (GR17, GR24, GR17) PROVIDED (S - 2.0)- 
GR17 TRANSFER (GR24, GR24, GR30) PROVIDED (S - FM)- 
GR24 P = PI- 

TRANSFER (GR100)- 
GR30 PS = PS + Pl- 
GR31 FMNIF = FM + FI- 

CALL SUBROUTINE ( ) = GAMMA. ( FMNIF, FM NIF)- 
FWH = (-l.O).P.(J-l) * FMNIF/ (FJJ * FIJJ * FMG)- 
SUM = SUM + FMM * PS- 
PS = 0.0- 
GR32 S = 0.0- 

B(I) = FK * BE.P.(I-I) * SUM- 
GR33 SUM =0.0- 

R = SQRT.(T) * W * (B(l( + W * (B(2) + W * (B(3) + W * 

( B(4) + W * (B(5)))))- 

NORMAL EXIT- 

GR200 CALL SUBROUTINE ( ) = ENDJOB. ( )- 
END SUBPROGRAM- 

SUBROUTINE ( ) = GAMMA. (Z,G2)- 
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II TERMS ( Cl , 0 . 0019 175269 18 , C2 , 0 . 0008417508418 , 

C3 , 0 . 00059 52381 , C4 , 0 . 00079 36507936508, C5 , 0 . 0027777777777 , 
C 6 , 0 . 08333333333 ,07,0.9189 358533 )- 

UPPER COMMON (IND)- 

BOOLEAN (IND)- 

X = Z- 

CV = 1.0- 

GA9 TRANSFER (GA10, GA11, GAll) PROVIDED (X -10.0)- 
GA10 CV = CV * X- 

X = X +1.0- 
TRANSFER (GA9 )- 

GAll TRANSFER (GA12, GA80, GA12) PROVIDED (CV)- 

GA12 TRANSFER (GA10, GA13. GA13) PROVIDED (.ABS.CV - 1.0)- 

GA 13 zz = 1.0/x/x- 

FX = (((((Cl * ZZ) ■+• C2) * ZZ - C3) * ZZ + C4) * ZZ - C5) 
* ZZ + C 6 - 

FX = FX/X- 

G = (X - 0.5) * LN.(X) + C7 - X + FX- 
TRANSFER (GA14, GA90, GA90) PROVIDED (G - 700.0)- 
GAl4 GZ = EXPE. (G)/CV- 

MORl'lAL EXIT- 

GA 80 WRITE OUTPUT, F 8 l, (Z)- 
F F81 O’+K GAKI-IA OF ZERO OR NEGATIVE INTEGER, E14.8)- 
IND(=).0. 777777777777- 
NORMAL EXIT- 

GA90 WRITE OUTPUT, P'91, (Z)- 

F F91 (44H GAMMA FUNCTION WILL EXCEED MACHINE CAPACITY, E24.8)- 

IND(=).0. 777777777777- 
NORMAL EXIT- 
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END SUBPROGRAM- 



SCATRAN SUBROUTINE 
FOR THE 

GARFINKEL REFRACTION MODEL 
U. S. COAST AND GEODETIC SURVEY VERSION 

SUBROUTINE (R) = REFGAR. (ZZ,PR,TT)- 

FLOATING (N1,N2,N3,N4)- 

UTERALS ( Nl, 1050. 61030, N2, 706. 11502, N3, 262. 06086, 
N4, 142. 67293. GAMMA, 8. 7137)- 

T = TT/273.16 = 1.0- 

P = PR/760.0 * 25.39998- 

W = P/(T * T)- 

TANB = (SQRT.(T) .GAMMA) * (SIN.(ZZ)/COS.(ZZ))- 
B = ATAN.(TANB)- 



BH = B/2.0- 



TANBH = SIN. ( BH)/COS .( BH)- 
FT = W * (SQRT.(T))- 

R = FT * ((Nl * TANBH ) + (N2 * (TANBH.P.3)) + 
(N3 * (TANBH. P.5)) + (N4 * ( TANBH. P.7)) )- 

NORMAL EXIT- 

END SUBPROGRAM- 
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SCATRAN SUBROUTINE 
FOR THE 

GARFINKEL REFRACTION MODEL 
ACIC FORM 

SUBROUTINE (REF) = REFGAR. ( ZEN ,PRS , TM)- 
DIKENSION (FNl(6l ),FN2(50),FN3( 15))- 

LITERALS (FN1 ,0.0,8.7,17.3.25.9,34.6,43.3,52.0,60.7,69.5. 

78.3.87.1.96.0. 104.9.113.8.122.9.131.9.141.0.150.2.159.6, 

168.9.178.3.187.8.197.4.207.1.216.9.226.9.237.0. 247.1, 
257.4,267.9,278.5,289.2,300.1,311.2,322.4,333.9,345.9, 

357.3. 369. 4. 381 .7. 394.2. 407. 0. 420. 0.433. 3.4,.9,450.8, 

475.0. 489.6.504.5.519.7.535.4.551.5.567.9.584.9.602.4, 
620.3,638.8,657.8,677.4,697.7,718.6)- 

LITERALS (FN2, 0.0, 0.0, 0.0, 0.1, 0.1, 0.1, 0.1, 0.1, 0.2, 0.2, 
0.2, 0.3, 0.3, 0.4, 0.4, 0.5, 0.6, 0.7, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 

1.4.1. 5 . 1.7. 1.9. 2. 1.2. 3. 2. 5. 2.7. 3. 0. 3. 3. 3.6. 3.9. 4.3. 4.7, 

5.1.5.5.5.9.6.5.7.0. 7.6.8.3.9.0.9.7.10.5.11.4.12.4,- 

LITERALS (FN3, 0.0, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.2, 0.2, 

0. 2, 0.2, 0.3, 0.3, 0.4)- 

TS = 273.0/ ( TM + 273.0)- 

T = SQRT.(TS)- 

GAMMA = 8.1578- 

P = (PRS/760.0) * 25.39998- 

W = P * TS * TS- 

COTZ = C0S.(ZEN)/SIN.(ZEN)- 

GAMO = GAMMA * T- 

TAN = 1 . 0/ ( GAMO * COTZ)- 

THETA = ATAN.(TAN)- 

ANG = THETA * 57.2957795- 

DO THROUGH (ONE), I=0,1,I.L.60- 

J = I + 1- 

PRO VIDEO ( ANG.L.I .OR.ANG.G. J. ) , TRANSFER (ONE)- 
B1 = FN1(I) + ((FN1(J) -FN1(I)) * (ANG - I))- 
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PROVIDED (I .LE. JO), TRANSFER (SETl)- 
IK + I - 11- 

JK = J - 11- 

B2 = FN2(IK) + ( ( FN2( JK) - FN2(IK)) * (ANG - I))- 
PROVIDED (I.LE.45), TRANSFER (SET2)- 
IM = I - 46- 

JM = J - 46- 

B3 + FN3(IM) + ((FN3(JM) - FN3(IM)) * (ANG - I))- 
TRANSFER (OUT)- 
SET1 B2 = 0.0- 

SET2 B3 = 0.0- 

TRANSFER (OUT)- 
ONE CONTE NUE- 

OUT REF = (W*B1)/T + (W * W * B2)/T + (W * W * W * B3)/T- 
NORMAL EXIT- 
END SUBPROGRAM- 
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SCATRAN PROGRAM 
FOR THE 

COMPARISON OF THE REFRACTION 
GIVEN BY THE THREE MODELS 
FOR VARIOUS TEMPERATURE 
AND PRESSURE COMBINATIONS 

DIMENSION (RM(54l),ZA(5M))- 

START READ IMPUT, USREF, ((RM(K), K=0, 1 , K.LE.540) )- 

F US REF (12F5.1)- 

DO THROUGH (TABLE), 1*0,1, 1. L. 540- 

DO THROUGH (TABLE), J=0,1.J.L.6- 

XJ = I + J- 

ZA(IJ) = (1/6.0) + (J * 0.166666667)- 
TABLE CONTI NUE- 

ZA(540) = 90.0- 
RHUM = 0 . 60 - 

DIMENSION (CENT(3).PRSMM(4))- 

LITERALS (CENT.-5. 0,10. 0,35.0)- 

LITERAIS (PRSMM, 711. 0,737.0, 760.0,787.0)- 

DO THROUGH (END), 1=0, 1,1. L. 3- 

CTEM = CENT(I)- 

DO THROUGH (END), J=0,1,J.L.4- 

WRITE OUTPUT, Fl- 

PRSIN = PRSMM (J)/25. 39998- 

DO THROUGH (ONE), K=0,1,K.L. 19- 

FACT = K- 

ZD = 5.0 * FACT- 

ZRAD = ZD/57.2957795- 
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ONE 

El® 
F Fl 

F F2 
F F3 



CALL SUBROUTINE (COAST) = REFCGS. (RHUM.CTEM.PRSIN.ZRAD)- 
CALL SUBROUTI NE (BAID) = REFBAL. (RHUM,CTEM,PRSIN,ZRAD)- 
CALL SUBROUTINE (GARF) = REFGAR. (ZRAD.PRSIN.CTEM)- 
WRITE OUTPUT, F2, ( ZD, COAST, BALD, GARF)- 
CONTI NUE- 

WRITE OUTPUT, F3, (CENT(I ) ,PRSMM( J) )- 
CONTI NUE- 

( 1H1 ,Q$ COMPARISON OF REFRACTION FROM 0 TO 90 DEGREES AT 
FIVE DEGREE INTERVALS$//Q$ ZENITH DISTANCE $,5X,Q$ COAST 
AI® GEODETIC SURVEY MODELS, 5X,Q$ BALDINI MODELS, 5X,Q$ 
GARFINKEL MODELS )- 

(//6X,F4.1,20X,F10.6,19X,F10.6,10X,F10.6)- 

(// Q$ ABOVE VALUES ARE IN SECONDS OF ARC$//QS THIS DATA 
WAS COMPUTED FOR A WAVE LENGTH OF 0.578 MICRONS$//Q$ 
RELATIVE HUMIDITY EQUALS 60 PER CENT$//Q$ CENTIGRADE 
TEMPERATURE EQUALSS,F7.l//QS ATMOSPHERIC PRESSURE EQUALS S , 
F7.1.1X.QS millimetersS)- 

CALL SUBROUTINE ( ) = ENDJOB. ( )- 

U. S. COAST AND GEODETIC SURVEY SUBROUTINE 

(MODIFIED FOR A LARGER RANGE OF TEMPERATURE AND PRESSURE) 

BALDINI SUBROUTINE 

GARFINKEL SUBROUTINE 

END PROGRAM (START)- 
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COMPARISON OF THE GARFTNKEL MODEL SUBROUTINES 



Zenith 
Distance 
(decrees ) 


Aberdeen 

Version 


Coast and 
Goodotic 
Survey 
Version 


Aberdeen 

Version 

Original 

Constants 


ACIC 

Form 


0 


0.000 


0.000 


0.000 


0.000 


5 


5.090 


5.088 


5.123 


5.157 


10 


10.257 


10.254 


10.325 


10.367 


15 


15.587 


15.582 


15.690 


15.702 


20 


21.171 


21.164 


21.311 


21.288 


25 


27.120 


27.112 


27.300 


27.266 


30 


33.574 


33.564 


33.796 


33.784 


35 


40.711 


40.699 


40.980 


40.979 


ho 


48.775 


48.760 


49.097 


49.091 


45 


58.107 


58.091 


58.492 


58.463 


50 


69.216 


69.198 


69.674 


69.685 


55 


82.887 


82.867 


83.435 


83.422 


60 


100.414 


100.393 


101 .079 


101.059 


65 


124.100 


124.801 


124.922 


124.922 


70 


158.465 


158.457 


159.418 


159.554 


75 


213.740 


213.770 


215.169 


215.100 


80 


318.633 


318.825 


320.830 


320.933 


85 


589.651 


590.955 


594.659 


594.666 



Temperature = 10° Centegrade 
Pressure = ?60,0 mm of Hg 

Refraction in Seconds of Arc 
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APPENDIX 

II 
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ADJUSTMENT OF THE REFRACTION MODEL 
FOR STATION CONDITIONS 



This adjustment procedure makes use of the Gonoral Method of 



Adjustments to obtain a solution for the variation of the assumed 

( 10 ) 



paramo tors . 
notation as 



The gonoral matrix form may be represented in matrix 



BF" 1 B* 


A 




FI 

tr* 

L 




w 


A' 


0 

—J 


1 


L x J 


i 


A 



or may bo given in a matrix equation as 

BV + AX + W = 0. 

In these forma all letters represent matrices (with the exception of 
the subscript). The A, B and VJ matrices are defined as 



A = A__F 
B = ^VF 
w = Fd^) 



where represents the observed quantities, represents the adjusted 
values of the observed quantities, X Q represents the assumod parameters 
and X_ represents the adjusted values for the parameters. Other matrices 

a 

used in the solution of the equations are the V matrix which gives the 
residuals for the observation, the X matrix which gives the corrections 
to the assumed paramotors and the P matrix which is the weight matrix. 

The prime symbols on the matrix indicate that it is the transpose of tho 
matrix whose letter is used and the minus one notation indicates that 
this matrix is the inverse of the matrix whose letter is used. De- 
fining a new matrix as 
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K = BP“ X B' 

tho final fom of the equation for the X matrix is given as 

X = -(A'M^A )" 1 A'H _1 W . 

For an adjustment to take place the condition must bo satisfied 

that 



n -f u r u 

where n is the number of observations of the observed quantities, u is 
the number of unknown parameters and r is the number of condition 
equations* When these conditions are fulfilled then tho adjustment 
can take place. 

The above factors and principles are thon an outline of the 
method which was used to attempt to adjust the refraction coefficients 
for the conditions at the stations. 

Two mathematical structures were employed in the adjustment 
attempt with two seperato attempts being used to make the adjustment. 

In the first instance the observed quantities were the observed al- 
titude and hour angle of each star. This gave four observed quantities 
per condition. In the second instance only two observed quantities 
were required, those being tho altitude and azimuth for the single star 
inquestion. In both cases the parameters were the three refraction 
model coefficients and tho assumed latitude. The declination, tempera- 
ture, pressure and time between direct and rovorse sightings of the 
instrument were considered not to vary for the purposes of the adjustment. 

The equation used for the refraction model was based on the 
ACIC form of tho Garfinkel Model which is given here as 

R = \/t£ W(N^K cot h o + N 2 K 3 cot 3 h o + N 3 K 5 cotfy 

vjhere 
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In this form P and 



'll = P/Tq 

K = t „A 



with Y assigned the Aberdeen value of 8.7137* 
T 0 are the relative pressures and temperatures, 
arc obtained from the relation 



Those quantities 



T = T + 273*16 
o o 

273.16 
p = p/ 760.0 



where the observed temperature is in degrees centigrade and the 
observed prossure is in millimeters of mercury. Tho cotangent func- 
tion appears in the equation since 

tan © = T 0 tan Z Q = T 0 cot h Q 

T T 

where Z Q is the observed zenith distance and h Q is the observed alti- 
tude. 



The relation may also be established that 

h = h D + R 

where h is the observed altitude after correction for the refraction 
and R is the refraction correction from the refraction model. These 
relationships will be used in the normal equation coefficients which 
are derived for each mathematical structure. 

The first mathematical structure investigated was based on the 
fact that, if all things wore perfect, the latitude obtained from an 
observation of a south circummcridian star would be tho samo as that 
obtained from the observation of Polaris. Ihis is fitting the more 
genoral statement that tho latitude obtained from tho north star should 

be the same as the latitude obtained from the south star to our avail- 
able data. Tho basic condition is then that the difference of these 
two values be equal to zero. This mathematical structure is given as 
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with the subscripts denoting the north and south stars. The reduction 
methods used by Thors on ^ were used to obtain these values in torms 
of the observed values and parameters. Using Polaris* the latitude 
from the north star observation becomes 

= h - P cos t + B 

whero 

P = (90° -S) 

• r2 2 

B = V tan h sin t 

with t representing the hour angle, representing the declination and 
h being the corrected altitude of Polaris, The latitude from the 
circunmoridian star may be represented as 

0 S = 6 - h* + 90° 

where 



h 1 = h + Am 

m = + m2 = 0.0001364( & T S ) 2 + sin 2 it 

A = cos 5> COS 0 
cos h 

with h representing the corrected altitude, t given as the hour angle 
of the south star, <5 representing the declination and & T s given as 
the time in seconds between the direct and reverse pointings of the 
instrument. Now denoting the north star quantities with an asterisk 
and the south star quantities with no superscript the mathematical 
structure may be written as 

h* - R* - P cos t* + i P tan (h* - R*) sin 2 t* 

o - £ +h _R + cos Scos 0 0.0001364(1-. I s )2 + 2 sin^g-t = 0. 

-90 ° CC-, 



90 



Since eight observations are taken of the north and south stars 
individually, this mathematical structure gives eight conditions to 
be fulfilled. There are four assumed parameters and a total of thirty- 
two observed quantities. This gives the values 

n « 32 
r - 8 
u » 4 

which when used to ascertain if an adjustment will take place gives 
the relation 

(32 + 4)>8>4. 

Therefore, an adjustment can exist for this amount of data used in the 
above mathematical structure. 

The A and B matrices are the ones whose form we are concerned 
with here. All abservations are taken to be of the same weight which 
causes the P matrix to be a unit matrix which may be dropped from the 
equations. It should be noted here that the values of the altitude and 
hour angle are correlated and that possible weights should be assigned 
to take account of this fact. The A matrix takes the form 

X r X f a f All 

_ X N 2 <X n 3 

and the 3 matrix is of the form 

A F F ^ F A r 

^ h 0 6t 0 cv ^ t-Q 

For simplification in the writing of the formulas for these partial 

derivatives some factors will be designated for use in the formulas. 
These factors are 

FR = N x \f^o \ + 3N 2 ( TO 5 cot2 o + 5N 3 ( cot4 ho 

sin 2 h 0 V d ^ Y j sin^liQ 



91 



FW = a\I T 

i c 



FK = '» J o cot h 

o 

V 

_ cos A cos o sin (h 0 ~ R) 
cos^(h — R) 

FPOL = sin 2 t* 

ec •> ^ o*- \V) 

FCIR =5 sin (h Q — R) 



cos (h Q - R) 



which vd.ll be usod along with the factors A and in defined above. 

The partial derivitaves for the A and B matricos nay now bo written 



Ahg 

hF 

At* 

AF 

AIL 

At 



1 + Fd(FR*) + FPOL [7 + FW( FR*)J 

P sin t* + P 2 tan (H* - R*) sin t* cos t* 



1 + FW(FR) + 

COS & COS 0 

cos (h Q — R) 



-L(h r) D - 

o 

(2 sin it cos i't) 



= — FW( FN* ) - FW( FK* ) FPOL - FW( FK ) ( Am )FCIR 
M~ = -FW(FN*P-FJ(FN*P FPOL-F//(FN) 3 -F.7(FNp(Am)FCIR 
j|— = -FW(FN*) 5 - F.7(FN*) 5 FPOL- FW( FN) 5 - FW( FN) 5 (Am)FCIR 



A F = -Am tan <£> 

where the valuos with the askorisk once again represent values computed 
fron the north star and those same values without a superscript represent 
these quantities computed from the south star. These derivitives are 
rather involved and a simplor system would be more desirablo if it 
could bo found. 

A simpler and more general method of expressing the mathematical 
structure is to use the first equation for the astronomical trianglo 
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given in Section 4.1, This gives the form 

sin p sin .<4 + cos C' cos S cos t — sin h = 0 
where tho quantities used in the formula are the same as those which 
were described above. In this method, each star observation must ful- 
fill the condition given by tho mathematical structure. 

For this mothod thorc are thirty two observed quantities, six 
teen conditions to be fulfilled and four unknowns. Tho data used for 
this method was tho same as in the case of the first mathematical 
structure given above. The assumed parameters remain the same in 
this adjustment and therefore there is no change in the general form 
of the A matrix. The 3 matrix is reduced to two terms since only the 
observation of a single star is considered for each condition. This 
lets the B matrix assume tho form 

~~ -4 F F ~j 

_~3T~ XK o J 

where t is now the hour angle of tho observed star and h the observed 

o 

altitude with no distinction being made for the north and south stars. 
For this mothod tho values 

n = 32 
r = 16 
u st 4 

are obtained which give tho relation 

(32 + 4) ■> 16 > 4. 

This relation shows that an adjustment can exist for this mathematical 
structure with the available data. 

The partial dorivitavos for this mothod aro easier to work with 
than thoso which wore taken from the previous mathematical structure. 
Using tho constant factors F>/, FiJ and FR from abovo the partial 
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derivatives bocono 



II 

&J-P 

A^ 


- cos ^ cos -b 


sin t 


e= 

AHq 


cos (h - 


- R) 


[”l + FW( FR) 


II 


-cos(h 0 — 


R) 


(FvJ) (FK) 




1 

o 

0 
cn 

ST 

O 

1 


R) 


(Fv7) (FN) 3 


hF _ 

^n 3 


1 

o 

0 
cn 

O 

1 


R) 


(FW) (FN) 5 


II 

fx* ISL 

sttfo 


cos / sin 


S - 


— sin cos § 



with theso values being computed for each observed star. 

In both of theso adjustment methods given above the W matrix 
can be computed from the mathematical structuro. Then with tho A, B 
and V7 matrices the matrix equations can bo used for each adjustment and 
a solution should be obtained if tho data are not too strongly 
correlated. 

The values for the refraction model coefficients wore selected 
as 

h = 522.5 
N 2 = 86.9 

Nj = 8.2 

from an examination of tho Garfinkel modol output for zenith distances 
from hO to 60 degroes. Tho numorical values given above aro an average 
of all tho coefficient values obtained by this process. 
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